[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:17 dist-test-slave-cbsf krb5kdc[24307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:17 dist-test-slave-cbsf krb5kdc[24307](info): set up 2 sockets
Aug 11 14:30:17 dist-test-slave-cbsf krb5kdc[24307](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 14:30:18.092343 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:18.226184 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37923
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:37923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:18.317981 24323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:18.318233 24323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:18.318259 24323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:18.319585 24323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:18.319639 24323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:18.319655 24323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:18.319669 24323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:18.319680 24323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:18.321574 24323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:37923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:18.321874 24323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:18.322136 24323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:18.327560 24329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.327560 24328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.327663 24331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:18.327791 24323 server_base.cc:1034] running on GCE node
I20260811 14:30:18.328167 24323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:18.328598 24323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:18.329823 24323 hybrid_clock.cc:648] HybridClock initialized: now 1786458618329831 us; error 65 us; skew 500 ppm
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:18.332474 24323 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:18.333115 24323 webserver.cc:460] Webserver started at http://127.23.186.254:35691/ using document root <none> and password file <none>
I20260811 14:30:18.333375 24323 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:18.333451 24323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:18.333616 24323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:18.334972 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "10a6b112164749b29ce04fda3f1b47ea"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "96192c0a3327f0f116782e41458f8fda"
server_key_iv: "7fe9249253d0da9d2c6743bc4d064e35"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.335412 24323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "10a6b112164749b29ce04fda3f1b47ea"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "96192c0a3327f0f116782e41458f8fda"
server_key_iv: "7fe9249253d0da9d2c6743bc4d064e35"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.337183 24323 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 14:30:18.338028 24339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.338254 24323 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:18.338378 24323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "10a6b112164749b29ce04fda3f1b47ea"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "96192c0a3327f0f116782e41458f8fda"
server_key_iv: "7fe9249253d0da9d2c6743bc4d064e35"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.338462 24323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:18.362926 24323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:18.365393 24323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:18.365590 24323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:18.370257 24323 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:37923
I20260811 14:30:18.370299 24399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:37923 every 8 connection(s)
I20260811 14:30:18.370759 24323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:18.371932 24400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:18.374590 24400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea: Bootstrap starting.
I20260811 14:30:18.374632 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 24323
I20260811 14:30:18.374788 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:18.375317 24400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:18.375484 24299 external_mini_cluster.cc:1428] Setting key bc330620190ddadb3c52046b6fa5a5f0
I20260811 14:30:18.375689 24400 log.cc:824] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:18.376645 24400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea: No bootstrap required, opened a new log
I20260811 14:30:18.378360 24400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a6b112164749b29ce04fda3f1b47ea" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37923 } }
I20260811 14:30:18.378540 24400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:18.378618 24400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10a6b112164749b29ce04fda3f1b47ea, State: Initialized, Role: FOLLOWER
I20260811 14:30:18.378777 24400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a6b112164749b29ce04fda3f1b47ea" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37923 } }
I20260811 14:30:18.378935 24400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:18.378976 24400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:18.379061 24400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:18.379994 24400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a6b112164749b29ce04fda3f1b47ea" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37923 } }
I20260811 14:30:18.380092 24400 leader_election.cc:302] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10a6b112164749b29ce04fda3f1b47ea; no voters:
I20260811 14:30:18.380299 24400 leader_election.cc:288] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:18.380368 24405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:18.380534 24405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [term 1 LEADER]: Becoming Leader. State: Replica: 10a6b112164749b29ce04fda3f1b47ea, State: Running, Role: LEADER
I20260811 14:30:18.380592 24400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:18.380788 24405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a6b112164749b29ce04fda3f1b47ea" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37923 } }
I20260811 14:30:18.381273 24406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10a6b112164749b29ce04fda3f1b47ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a6b112164749b29ce04fda3f1b47ea" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37923 } } }
I20260811 14:30:18.381371 24406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:18.381598 24407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10a6b112164749b29ce04fda3f1b47ea. Latest consensus state: current_term: 1 leader_uuid: "10a6b112164749b29ce04fda3f1b47ea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10a6b112164749b29ce04fda3f1b47ea" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37923 } } }
I20260811 14:30:18.381865 24407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:18.382071 24415 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:18.382704 24415 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:18.385077 24415 catalog_manager.cc:1347] Generated new cluster ID: ffe771e0ac4c4af49983e76198790737
I20260811 14:30:18.385139 24415 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:18.392920 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.376562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47858 (local address 127.23.186.254:37923)
0811 14:30:18.378660 (+ 2098us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:18.378671 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:18.378706 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:30:18.378822 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:18.378828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:18.378906 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:18.379084 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:18.381142 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.382654 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.383983 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.384628 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.384780 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.388433 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:18.388438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:18.388439 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:18.388454 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:18.390771 (+ 2317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.391265 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.391269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.391269 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.391379 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.391669 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.391671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.391671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.391932 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:18.392123 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:18.392414 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 14:30:18.392577 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2040,"thread_start_us":92,"threads_started":1}
I20260811 14:30:18.393034 24415 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:18.393719 24415 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:18.399365 24415 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10a6b112164749b29ce04fda3f1b47ea: Generated new TSK 0
I20260811 14:30:18.399621 24415 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:18.436308 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37923
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:18.547400 24428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:18.547636 24428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:18.547663 24428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:18.548949 24428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:18.548995 24428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:18.549026 24428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:18.550598 24428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37923
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24428
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:18.550863 24428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:18.551126 24428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:18.556457 24433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.556471 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.556504 24436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:18.557159 24428 server_base.cc:1034] running on GCE node
I20260811 14:30:18.557374 24428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:18.557603 24428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:18.558751 24428 hybrid_clock.cc:648] HybridClock initialized: now 1786458618558737 us; error 31 us; skew 500 ppm
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:18.561667 24428 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:18.562110 24428 webserver.cc:460] Webserver started at http://127.23.186.193:41891/ using document root <none> and password file <none>
I20260811 14:30:18.562304 24428 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:18.562376 24428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:18.562513 24428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:18.563647 24428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "3422307518c14418a0a95e6d890f3563"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "b1e4a7b652989e97b17a9a5f997312be"
server_key_iv: "2558cc34a42c3517a7f3c2335a83524f"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.563961 24428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "3422307518c14418a0a95e6d890f3563"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "b1e4a7b652989e97b17a9a5f997312be"
server_key_iv: "2558cc34a42c3517a7f3c2335a83524f"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.565471 24428 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:18.566361 24445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.566584 24428 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:18.566713 24428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "3422307518c14418a0a95e6d890f3563"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "b1e4a7b652989e97b17a9a5f997312be"
server_key_iv: "2558cc34a42c3517a7f3c2335a83524f"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.566802 24428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:18.595196 24428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:18.598124 24428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:18.598328 24428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:18.598618 24428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:18.599001 24428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:18.599068 24428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.599115 24428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:18.599164 24428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.606526 24428 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:44953
I20260811 14:30:18.606606 24578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:44953 every 8 connection(s)
I20260811 14:30:18.607005 24428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:18.614113 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 24428
I20260811 14:30:18.614224 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:18.614389 24299 external_mini_cluster.cc:1428] Setting key 9bce8d9c78b2b4bd9b50b075b3593894
I20260811 14:30:18.618249 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.607841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:49593 (local address 127.23.186.254:37923)
0811 14:30:18.607924 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:18.607926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:18.608173 (+ 247us) server_negotiation.cc:407] Connection header received
0811 14:30:18.608799 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:18.608800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:18.608816 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:18.608881 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:18.610075 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.610618 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.611721 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.611932 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.611991 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.614588 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:18.614590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:18.614590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:18.614599 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:18.616301 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.616940 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.616942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.616942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.617006 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.617407 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.617408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.617408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.617583 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:18.617660 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:18.617994 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 14:30:18.618122 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:30:18.618278 24581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.607831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37923 (local address 127.23.186.193:49593)
0811 14:30:18.608018 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:18.608066 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:18.608628 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:18.608948 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:18.608952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:18.609709 (+ 757us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:18.609941 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:18.609947 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.610749 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.610750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:18.611631 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:18.611634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.612038 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.612039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:18.612177 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.612788 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:18.612805 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:18.614411 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:18.616404 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:18.616415 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:18.616418 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:18.616853 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:18.617090 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:18.617091 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:18.617091 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:18.617302 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:18.617673 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:18.617681 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:18.617826 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 14:30:18.617918 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:30:18.618071 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260811 14:30:18.618911 24579 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37923
I20260811 14:30:18.619041 24579 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:18.619242 24579 heartbeater.cc:511] Master 127.23.186.254:37923 requested a full tablet report, sending...
I20260811 14:30:18.619845 24360 ts_manager.cc:194] Registered new tserver with Master: 3422307518c14418a0a95e6d890f3563 (127.23.186.193:44953)
I20260811 14:30:18.620576 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:49593
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:18.644567 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37923
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:18.731977 24584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:18.732203 24584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:18.732231 24584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:18.733414 24584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:18.733454 24584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:18.733481 24584 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:18.734943 24584 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37923
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:18.735163 24584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:18.735350 24584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:18.740468 24589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.740496 24591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.740489 24593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:18.741165 24584 server_base.cc:1034] running on GCE node
I20260811 14:30:18.741390 24584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:18.741657 24584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:18.742805 24584 hybrid_clock.cc:648] HybridClock initialized: now 1786458618742794 us; error 31 us; skew 500 ppm
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:18.745711 24584 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:18.746241 24584 webserver.cc:460] Webserver started at http://127.23.186.194:37437/ using document root <none> and password file <none>
I20260811 14:30:18.746495 24584 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:18.746572 24584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:18.746728 24584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:18.747846 24584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "53376780f7e14a9bb700963bbbb20399"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "39652914972fb78f1b4191cf08caafbf"
server_key_iv: "e5e8f6b99d9f43c31249cfc7149f9d6d"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.748238 24584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "53376780f7e14a9bb700963bbbb20399"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "39652914972fb78f1b4191cf08caafbf"
server_key_iv: "e5e8f6b99d9f43c31249cfc7149f9d6d"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.749840 24584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:18.750717 24602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.750981 24584 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:18.751106 24584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "53376780f7e14a9bb700963bbbb20399"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "39652914972fb78f1b4191cf08caafbf"
server_key_iv: "e5e8f6b99d9f43c31249cfc7149f9d6d"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.751189 24584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:18.773545 24584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:18.776054 24584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:18.776232 24584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:18.776499 24584 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:18.776849 24584 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:18.776902 24584 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.776944 24584 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:18.776968 24584 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.783684 24584 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33369
I20260811 14:30:18.783731 24736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33369 every 8 connection(s)
I20260811 14:30:18.784107 24584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:18.791431 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 24584
I20260811 14:30:18.791530 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:18.791759 24299 external_mini_cluster.cc:1428] Setting key 134f033ebd059da5316bbbe522e08595
I20260811 14:30:18.794175 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.785076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:50609 (local address 127.23.186.254:37923)
0811 14:30:18.785156 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:18.785158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:18.785368 (+ 210us) server_negotiation.cc:407] Connection header received
0811 14:30:18.785872 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:18.785873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:18.785883 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:18.785948 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:18.786929 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.787436 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.788439 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.788632 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.788686 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.791012 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:18.791014 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:18.791014 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:18.791022 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:18.792670 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.793109 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.793110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.793111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.793177 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.793416 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.793417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.793417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.793575 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:18.793644 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:18.793906 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 14:30:18.794025 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:30:18.794188 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.785068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37923 (local address 127.23.186.194:50609)
0811 14:30:18.785230 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:18.785277 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:18.785783 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:18.786006 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:18.786009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:18.786625 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:18.786838 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:18.786844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.787529 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.787529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:18.788354 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:18.788356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.788758 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.788759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:18.788879 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.789470 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:18.789480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:18.790912 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:18.792782 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:18.792789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:18.792791 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:18.793025 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:18.793247 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:18.793247 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:18.793248 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:18.793352 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:18.793638 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:18.793642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:18.793774 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 14:30:18.793863 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 14:30:18.794009 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260811 14:30:18.794948 24737 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37923
I20260811 14:30:18.795069 24737 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:18.795300 24737 heartbeater.cc:511] Master 127.23.186.254:37923 requested a full tablet report, sending...
I20260811 14:30:18.795778 24360 ts_manager.cc:194] Registered new tserver with Master: 53376780f7e14a9bb700963bbbb20399 (127.23.186.194:33369)
I20260811 14:30:18.796187 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:50609
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:18.823465 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37923
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:18.916059 24742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:18.916291 24742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:18.916317 24742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:18.917677 24742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:18.917734 24742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:18.917765 24742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:18.919368 24742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37923
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:18.919662 24742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:18.919924 24742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:18.925355 24747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.925338 24750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:18.925420 24748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:18.925887 24742 server_base.cc:1034] running on GCE node
I20260811 14:30:18.926141 24742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:18.926407 24742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:18.927582 24742 hybrid_clock.cc:648] HybridClock initialized: now 1786458618927566 us; error 43 us; skew 500 ppm
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:18.930436 24742 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:18.930958 24742 webserver.cc:460] Webserver started at http://127.23.186.195:46599/ using document root <none> and password file <none>
I20260811 14:30:18.931221 24742 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:18.931293 24742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:18.931457 24742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:18.932582 24742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "933343bbf1ee4a8488735de358940bae"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "bdb1c9d884636ed368d5e9d6270d7c65"
server_key_iv: "15608afb9fc1138eb6a2dc178a3e0e1c"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.933061 24742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "933343bbf1ee4a8488735de358940bae"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "bdb1c9d884636ed368d5e9d6270d7c65"
server_key_iv: "15608afb9fc1138eb6a2dc178a3e0e1c"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.934545 24742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:18.935336 24760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.935523 24742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:18.935637 24742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "933343bbf1ee4a8488735de358940bae"
format_stamp: "Formatted at 2026-08-11 14:30:18 on dist-test-slave-cbsf"
server_key: "bdb1c9d884636ed368d5e9d6270d7c65"
server_key_iv: "15608afb9fc1138eb6a2dc178a3e0e1c"
server_key_version: "encryptionkey@0"
I20260811 14:30:18.935715 24742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:18.946494 24742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:18.948796 24742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:18.948997 24742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:18.949281 24742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:18.949657 24742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:18.949713 24742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.949750 24742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:18.949802 24742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:18.956631 24742 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:40261
I20260811 14:30:18.956673 24893 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:40261 every 8 connection(s)
I20260811 14:30:18.957158 24742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:18.960048 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 24742
I20260811 14:30:18.960122 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:18.960296 24299 external_mini_cluster.cc:1428] Setting key 979be3f2ae4944f942ffc3fc0d27564f
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:18.967031 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.958098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:60623 (local address 127.23.186.254:37923)
0811 14:30:18.958189 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:18.958192 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:18.958383 (+ 191us) server_negotiation.cc:407] Connection header received
0811 14:30:18.958877 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:18.958878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:18.958889 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:18.958943 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:18.959956 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.960566 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.961905 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.962114 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.962195 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.964382 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:18.964383 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:18.964383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:18.964391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:18.965673 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.966070 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.966071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.966071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.966133 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:18.966392 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:18.966393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:18.966393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:18.966521 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:18.966581 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:18.966865 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 14:30:18.966963 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 14:30:18.967150 24896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.958066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37923 (local address 127.23.186.195:60623)
0811 14:30:18.958258 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:18.958296 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:18.958785 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:18.959000 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:18.959003 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:18.959627 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:18.959837 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:18.959864 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.960668 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.960669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:18.961812 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:18.961815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.962204 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.962205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:18.962344 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:18.962860 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:18.962869 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:18.964292 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:18.965746 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:18.965752 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:18.965754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:18.965983 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:18.966213 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:18.966214 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:18.966214 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:18.966324 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:18.966581 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:18.966585 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:18.966713 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 14:30:18.966803 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 14:30:18.966945 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
I20260811 14:30:18.967742 24894 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37923
I20260811 14:30:18.967873 24894 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:18.968096 24894 heartbeater.cc:511] Master 127.23.186.254:37923 requested a full tablet report, sending...
I20260811 14:30:18.968596 24360 ts_manager.cc:194] Registered new tserver with Master: 933343bbf1ee4a8488735de358940bae (127.23.186.195:40261)
I20260811 14:30:18.969053 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:60623
I20260811 14:30:18.972535 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:30:18 dist-test-slave-cbsf krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458618, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:19.006862 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:18.990731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52840 (local address 127.23.186.193:44953)
0811 14:30:18.990975 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:18.990980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:18.990997 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:30:18.991025 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:18.991027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:18.991041 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:18.991131 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:18.991937 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.992751 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.993531 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:18.993880 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:18.993968 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:19.002731 (+ 8763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:19.002753 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:19.002754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:19.002768 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:19.005663 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:19.006078 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:19.006082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:19.006082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:19.006144 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:19.006316 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:19.006317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:19.006317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:19.006472 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:19.006599 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:19.006662 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 14:30:19.006735 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":153,"threads_started":1}
W20260811 14:30:19.007315 24481 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52840
Aug 11 14:30:19 dist-test-slave-cbsf krb5kdc[24307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:30:19 dist-test-slave-cbsf krb5kdc[24307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458619, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:19.032919 24902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:19.025514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.23.186.193:44953)
0811 14:30:19.025624 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:19.025626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:19.025689 (+ 63us) server_negotiation.cc:407] Connection header received
0811 14:30:19.025743 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:19.025744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:19.025759 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:19.025812 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:19.026426 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:19.026955 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:19.027688 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:19.027891 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:19.027944 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:19.030399 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:19.030401 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:19.030401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:19.030410 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:19.031774 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:19.032149 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:19.032150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:19.032150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:19.032209 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:19.032386 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:19.032387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:19.032387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:19.032489 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:19.032610 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:19.032673 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 14:30:19.032811 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 14:30:19.033567 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 24428
I20260811 14:30:19.042797 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 24584
I20260811 14:30:19.054730 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 24742
I20260811 14:30:19.062258 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 24323
2026-08-11T14:30:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2136 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:19 dist-test-slave-cbsf krb5kdc[24909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:19 dist-test-slave-cbsf krb5kdc[24909](info): set up 2 sockets
Aug 11 14:30:19 dist-test-slave-cbsf krb5kdc[24909](info): commencing operation
krb5kdc: starting...
W20260811 14:30:20.197248 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:20.332226 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:36945
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:36945
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:20.419456 24922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:20.419690 24922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:20.419714 24922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:20.420913 24922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:20.420955 24922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:20.420969 24922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:20.420982 24922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:20.420994 24922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:20.422494 24922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:36945
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:36945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:20.422715 24922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:20.422904 24922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:20.427896 24927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:20.427949 24932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:20.427891 24930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:20.428257 24922 server_base.cc:1034] running on GCE node
I20260811 14:30:20.428517 24922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:20.428879 24922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:20.430050 24922 hybrid_clock.cc:648] HybridClock initialized: now 1786458620430039 us; error 42 us; skew 500 ppm
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:20.437909 24922 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:20.438498 24922 webserver.cc:460] Webserver started at http://127.23.186.254:43777/ using document root <none> and password file <none>
I20260811 14:30:20.438757 24922 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:20.438831 24922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:20.438995 24922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:20.440160 24922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "c84b5a048acc4cc8a445c464d0d00b63"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "9c49613e83060274baa0e6a3afbe3def"
server_key_iv: "5bfe55cff402ec8f3e734367f48eafaf"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.440541 24922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "c84b5a048acc4cc8a445c464d0d00b63"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "9c49613e83060274baa0e6a3afbe3def"
server_key_iv: "5bfe55cff402ec8f3e734367f48eafaf"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.442081 24922 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:20.442866 24939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.443058 24922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:20.443178 24922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "c84b5a048acc4cc8a445c464d0d00b63"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "9c49613e83060274baa0e6a3afbe3def"
server_key_iv: "5bfe55cff402ec8f3e734367f48eafaf"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.443262 24922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:20.474144 24922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:20.476310 24922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:20.476509 24922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:20.480755 24922 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:36945
I20260811 14:30:20.480846 24996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:36945 every 8 connection(s)
I20260811 14:30:20.481247 24922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:20.482262 24997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:20.484856 24997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63: Bootstrap starting.
I20260811 14:30:20.485590 24997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:20.485924 24997 log.cc:824] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:20.486784 24997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63: No bootstrap required, opened a new log
I20260811 14:30:20.488427 24997 raft_consensus.cc:348] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84b5a048acc4cc8a445c464d0d00b63" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36945 } }
I20260811 14:30:20.488588 24997 raft_consensus.cc:374] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:20.488613 24997 raft_consensus.cc:729] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84b5a048acc4cc8a445c464d0d00b63, State: Initialized, Role: FOLLOWER
I20260811 14:30:20.488826 24997 consensus_queue.cc:260] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84b5a048acc4cc8a445c464d0d00b63" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36945 } }
I20260811 14:30:20.488899 24997 raft_consensus.cc:388] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:20.488922 24997 raft_consensus.cc:482] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:20.488952 24997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:20.489233 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 24922
I20260811 14:30:20.489331 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:20.489488 24299 external_mini_cluster.cc:1428] Setting key b6634b14a92c285e908acc89859417c5
I20260811 14:30:20.489693 24997 raft_consensus.cc:504] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84b5a048acc4cc8a445c464d0d00b63" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36945 } }
I20260811 14:30:20.489820 24997 leader_election.cc:302] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c84b5a048acc4cc8a445c464d0d00b63; no voters:
I20260811 14:30:20.490027 24997 leader_election.cc:288] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:20.490054 25003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:20.490207 25003 raft_consensus.cc:686] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [term 1 LEADER]: Becoming Leader. State: Replica: c84b5a048acc4cc8a445c464d0d00b63, State: Running, Role: LEADER
I20260811 14:30:20.490304 25003 consensus_queue.cc:237] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84b5a048acc4cc8a445c464d0d00b63" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36945 } }
I20260811 14:30:20.490388 24997 sys_catalog.cc:564] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:20.490686 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c84b5a048acc4cc8a445c464d0d00b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84b5a048acc4cc8a445c464d0d00b63" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36945 } } }
I20260811 14:30:20.490765 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:20.490891 25003 sys_catalog.cc:455] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c84b5a048acc4cc8a445c464d0d00b63. Latest consensus state: current_term: 1 leader_uuid: "c84b5a048acc4cc8a445c464d0d00b63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c84b5a048acc4cc8a445c464d0d00b63" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36945 } } }
I20260811 14:30:20.490947 25003 sys_catalog.cc:458] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:20.491367 25011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:20.492161 25011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:20.494868 25011 catalog_manager.cc:1347] Generated new cluster ID: d038d08e13e7411f951ad06176066166
I20260811 14:30:20.494937 25011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:20.502336 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:20.490044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55454 (local address 127.23.186.254:36945)
0811 14:30:20.490369 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:20.490380 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:20.490414 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:30:20.490640 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:20.490648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:20.490722 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:20.491915 (+ 1193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:20.492043 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.493348 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.494444 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.495017 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.495155 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:20.497886 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:20.497890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:20.497891 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:20.497904 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:20.499798 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:20.500926 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:20.500938 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:20.500939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:20.501066 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:20.501322 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:20.501323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:20.501323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:20.501617 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:20.501802 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:20.501922 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 14:30:20.502101 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":109,"threads_started":1}
I20260811 14:30:20.506395 25011 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:20.507068 25011 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:20.512805 25011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c84b5a048acc4cc8a445c464d0d00b63: Generated new TSK 0
I20260811 14:30:20.513057 25011 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:20.551939 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36945
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:20.664136 25032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:20.664386 25032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:20.664422 25032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:20.665773 25032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:20.665835 25032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:20.665867 25032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:20.667582 25032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36945
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:20.667833 25032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:20.668048 25032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:20.673514 25038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:20.673656 25032 server_base.cc:1034] running on GCE node
W20260811 14:30:20.673552 25041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:20.673666 25039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:20.674067 25032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:20.674343 25032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:20.675526 25032 hybrid_clock.cc:648] HybridClock initialized: now 1786458620675505 us; error 37 us; skew 500 ppm
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:20.678610 25032 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:20.679175 25032 webserver.cc:460] Webserver started at http://127.23.186.193:33455/ using document root <none> and password file <none>
I20260811 14:30:20.679447 25032 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:20.679497 25032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:20.679656 25032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:20.680886 25032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "b0d0377b46194e3da4bf4c756d87d7a7"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "64195c638b19a5e3e55da07251cdef60"
server_key_iv: "3e9fba31b1ad594012fe67874fbae790"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.681272 25032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "b0d0377b46194e3da4bf4c756d87d7a7"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "64195c638b19a5e3e55da07251cdef60"
server_key_iv: "3e9fba31b1ad594012fe67874fbae790"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.682723 25032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:20.683584 25048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.683822 25032 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:20.683934 25032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "b0d0377b46194e3da4bf4c756d87d7a7"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "64195c638b19a5e3e55da07251cdef60"
server_key_iv: "3e9fba31b1ad594012fe67874fbae790"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.684013 25032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:20.700208 25032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:20.702620 25032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:20.702790 25032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:20.703083 25032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:20.703464 25032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:20.703504 25032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.703529 25032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:20.703541 25032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.710057 25032 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:43613
I20260811 14:30:20.710151 25188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:43613 every 8 connection(s)
I20260811 14:30:20.710582 25032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:20.719806 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25032
I20260811 14:30:20.719936 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:20.720126 24299 external_mini_cluster.cc:1428] Setting key 4e337649a1338fc9cf778a587be7c54a
I20260811 14:30:20.721135 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:20.711377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:43979 (local address 127.23.186.254:36945)
0811 14:30:20.711430 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:20.711433 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:20.711656 (+ 223us) server_negotiation.cc:407] Connection header received
0811 14:30:20.712905 (+ 1249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:20.712906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:20.712921 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:20.712986 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:20.713707 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.714227 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.715290 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.715505 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.715606 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:20.718113 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:20.718115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:20.718115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:20.718132 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:20.719453 (+ 1321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:20.719922 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:20.719924 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:20.719924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:20.720020 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:20.720322 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:20.720323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:20.720323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:20.720494 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:20.720559 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:20.720954 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 14:30:20.721067 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:20.721513 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:20.711327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:36945 (local address 127.23.186.193:43979)
0811 14:30:20.711521 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:20.711561 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:20.712763 (+ 1202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:20.713068 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:20.713072 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:20.713367 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:20.713584 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:20.713588 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.714333 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.714334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:20.715190 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:20.715192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.715630 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.715631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:20.715774 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:20.716323 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:20.716332 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:20.718005 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:20.719554 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:20.719564 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:20.719567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:20.719822 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:20.720099 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:20.720100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:20.720101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:20.720258 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:20.720560 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:20.720569 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:20.720783 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 14:30:20.720907 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 14:30:20.721137 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":76,"threads_started":1}
I20260811 14:30:20.722159 25189 heartbeater.cc:347] Connected to a master server at 127.23.186.254:36945
I20260811 14:30:20.722290 25189 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:20.722503 25189 heartbeater.cc:511] Master 127.23.186.254:36945 requested a full tablet report, sending...
I20260811 14:30:20.723102 24958 ts_manager.cc:194] Registered new tserver with Master: b0d0377b46194e3da4bf4c756d87d7a7 (127.23.186.193:43613)
I20260811 14:30:20.723778 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:43979
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:20.749771 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36945
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:20.834947 25194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:20.835184 25194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:20.835210 25194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:20.836464 25194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:20.836522 25194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:20.836552 25194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:20.838238 25194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36945
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:20.838531 25194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:20.838780 25194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:20.844189 25199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:20.844213 25200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:20.844203 25202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:20.844276 25194 server_base.cc:1034] running on GCE node
I20260811 14:30:20.844659 25194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:20.844995 25194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:20.846185 25194 hybrid_clock.cc:648] HybridClock initialized: now 1786458620846165 us; error 36 us; skew 500 ppm
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:20.849038 25194 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:20.849519 25194 webserver.cc:460] Webserver started at http://127.23.186.194:46667/ using document root <none> and password file <none>
I20260811 14:30:20.849787 25194 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:20.849861 25194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:20.850030 25194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:20.851152 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "14630222ef5f41918541f29da92d0a3f"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "1147d7178a3cd2b5299439dac4b34efa"
server_key_iv: "da4143e9eda6c0fa46a0cd462cf4bc4d"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.851527 25194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "14630222ef5f41918541f29da92d0a3f"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "1147d7178a3cd2b5299439dac4b34efa"
server_key_iv: "da4143e9eda6c0fa46a0cd462cf4bc4d"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.853019 25194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:20.853842 25210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.854051 25194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:20.854173 25194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "14630222ef5f41918541f29da92d0a3f"
format_stamp: "Formatted at 2026-08-11 14:30:20 on dist-test-slave-cbsf"
server_key: "1147d7178a3cd2b5299439dac4b34efa"
server_key_iv: "da4143e9eda6c0fa46a0cd462cf4bc4d"
server_key_version: "encryptionkey@0"
I20260811 14:30:20.854251 25194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:20.878532 25194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:20.881330 25194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:20.881541 25194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:20.881841 25194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:20.882221 25194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:20.882287 25194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.882339 25194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:20.882373 25194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:20.889359 25194 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:45763
I20260811 14:30:20.889418 25340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:45763 every 8 connection(s)
I20260811 14:30:20.889874 25194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:20.896346 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25194
I20260811 14:30:20.896454 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:20.896670 24299 external_mini_cluster.cc:1428] Setting key 3b6dfd3da016f89f03be13f0ee9964d0
Aug 11 14:30:20 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:20.901664 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:20.891088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:57515 (local address 127.23.186.254:36945)
0811 14:30:20.891160 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:20.891162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:20.891174 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:20.892464 (+ 1290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:20.892466 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:20.892479 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:20.892550 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:20.893376 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.893891 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.894993 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.895195 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.895256 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:20.897909 (+ 2653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:20.897911 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:20.897912 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:20.897921 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:20.899812 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:20.900390 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:20.900392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:20.900392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:20.900474 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:20.900833 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:20.900835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:20.900835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:20.901020 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:20.901094 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:20.901506 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 14:30:20.901593 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:30:20.902006 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:20.890775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:36945 (local address 127.23.186.194:57515)
0811 14:30:20.891024 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:20.891085 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:20.892321 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:20.892641 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:20.892645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:20.893035 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:20.893259 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:20.893264 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.893984 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.893984 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:20.894889 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:20.894891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:20.895300 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:20.895302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:20.895459 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:20.896046 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:20.896055 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:20.897799 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:20.899913 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:20.899925 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:20.899928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:20.900301 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:20.900545 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:20.900546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:20.900547 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:20.900759 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:20.901102 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:20.901109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:20.901301 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 14:30:20.901418 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 14:30:20.901633 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":79,"threads_started":1}
I20260811 14:30:20.902634 25341 heartbeater.cc:347] Connected to a master server at 127.23.186.254:36945
I20260811 14:30:20.902771 25341 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:20.902992 25341 heartbeater.cc:511] Master 127.23.186.254:36945 requested a full tablet report, sending...
I20260811 14:30:20.903424 24958 ts_manager.cc:194] Registered new tserver with Master: 14630222ef5f41918541f29da92d0a3f (127.23.186.194:45763)
I20260811 14:30:20.903836 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:57515
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:20.929531 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36945
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:21.021443 25347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:21.021716 25347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:21.021763 25347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:21.022985 25347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:21.023056 25347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:21.023103 25347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:21.024770 25347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36945
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:21.025061 25347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:21.025259 25347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:21.031101 25357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:21.031126 25354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:21.031219 25347 server_base.cc:1034] running on GCE node
W20260811 14:30:21.031101 25352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:21.031627 25347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:21.031898 25347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:21.033080 25347 hybrid_clock.cc:648] HybridClock initialized: now 1786458621033066 us; error 45 us; skew 500 ppm
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458621, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:21.035725 25347 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:21.036222 25347 webserver.cc:460] Webserver started at http://127.23.186.195:36017/ using document root <none> and password file <none>
I20260811 14:30:21.036479 25347 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:21.036551 25347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:21.036756 25347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:21.037909 25347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "cc6c99ddaa174744a19c7897b8ff1068"
format_stamp: "Formatted at 2026-08-11 14:30:21 on dist-test-slave-cbsf"
server_key: "c1866a40ec7caed0720074ebd7705ed5"
server_key_iv: "73a16a0ae02a4a83ff8ea879313876f0"
server_key_version: "encryptionkey@0"
I20260811 14:30:21.038309 25347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "cc6c99ddaa174744a19c7897b8ff1068"
format_stamp: "Formatted at 2026-08-11 14:30:21 on dist-test-slave-cbsf"
server_key: "c1866a40ec7caed0720074ebd7705ed5"
server_key_iv: "73a16a0ae02a4a83ff8ea879313876f0"
server_key_version: "encryptionkey@0"
I20260811 14:30:21.039741 25347 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:21.040587 25364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:21.040787 25347 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:21.040900 25347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "cc6c99ddaa174744a19c7897b8ff1068"
format_stamp: "Formatted at 2026-08-11 14:30:21 on dist-test-slave-cbsf"
server_key: "c1866a40ec7caed0720074ebd7705ed5"
server_key_iv: "73a16a0ae02a4a83ff8ea879313876f0"
server_key_version: "encryptionkey@0"
I20260811 14:30:21.040979 25347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:21.057210 25347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:21.059352 25347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:21.059535 25347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:21.059818 25347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:21.060168 25347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:21.060227 25347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:21.060271 25347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:21.060316 25347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:21.066668 25347 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:44949
I20260811 14:30:21.066720 25493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:44949 every 8 connection(s)
I20260811 14:30:21.067080 25347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458621, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:21.076133 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25347
I20260811 14:30:21.076253 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:21.076442 24299 external_mini_cluster.cc:1428] Setting key ebac406ac65684fa582a5ec1fd5a74ff
I20260811 14:30:21.076993 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.067889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:49843 (local address 127.23.186.254:36945)
0811 14:30:21.067958 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.067960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.068192 (+ 232us) server_negotiation.cc:407] Connection header received
0811 14:30:21.069259 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.069261 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.069274 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.069334 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:21.070004 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.070542 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.071597 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.071804 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.071894 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.074204 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:21.074205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:21.074205 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:21.074213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:21.075459 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.075853 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.075854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.075855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.075911 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.076188 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.076189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.076190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.076352 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:21.076427 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.076804 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.076902 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:30:21.077231 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.067845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:36945 (local address 127.23.186.195:49843)
0811 14:30:21.068059 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:21.068098 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:21.069131 (+ 1033us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:21.069414 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:21.069418 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:21.069697 (+ 279us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:21.069898 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.069902 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.070653 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.070655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.071478 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.071481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.071906 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.071908 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.072041 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.072630 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:21.072639 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:21.074112 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:21.075537 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:21.075544 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:21.075546 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:21.075787 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:21.076012 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:21.076013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:21.076013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:21.076122 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:21.076422 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:21.076428 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:21.076629 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 14:30:21.076778 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 14:30:21.076956 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260811 14:30:21.077703 25494 heartbeater.cc:347] Connected to a master server at 127.23.186.254:36945
I20260811 14:30:21.077826 25494 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:21.078017 25494 heartbeater.cc:511] Master 127.23.186.254:36945 requested a full tablet report, sending...
I20260811 14:30:21.078435 24958 ts_manager.cc:194] Registered new tserver with Master: cc6c99ddaa174744a19c7897b8ff1068 (127.23.186.195:44949)
I20260811 14:30:21.078836 24958 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:49843
I20260811 14:30:21.088584 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:30:21.096197 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.090367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55458 (local address 127.23.186.254:36945)
0811 14:30:21.090437 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.090439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.090494 (+ 55us) server_negotiation.cc:407] Connection header received
0811 14:30:21.090540 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.090541 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.090552 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.090615 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:21.091265 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.091758 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.092488 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.092673 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.092781 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.093994 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:21.093995 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:21.093995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:21.094002 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:21.095219 (+ 1217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.095532 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.095533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.095533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.095584 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.095755 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.095756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.095756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.095904 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:21.095988 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.096068 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.096131 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:30:21.098555 24958 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:21.100111 24958 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:21.107012 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.102999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34208 (local address 127.23.186.194:45763)
0811 14:30:21.103331 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.103335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.103347 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:21.103455 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.103457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.103472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.103552 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:21.103715 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.104832 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.106067 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.106585 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.106679 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.106745 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.106878 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.106920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":87,"threads_started":1}
I20260811 14:30:21.107088 25509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.102703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:43613 (local address 127.0.0.1:53870)
0811 14:30:21.102892 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:21.102935 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:21.103024 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:21.103328 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:21.103331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:21.103333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:21.103415 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.103417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.104477 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.104478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.105825 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.105827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.106774 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.106775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.106841 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.106844 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:21.106874 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 14:30:21.106930 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":62,"threads_started":1}
I20260811 14:30:21.107147 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.102956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39806 (local address 127.23.186.195:44949)
0811 14:30:21.103430 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.103434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.103443 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:21.103478 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.103480 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.103497 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.103570 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:21.103852 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.104904 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.105952 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.106672 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.106748 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.106803 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.107030 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.107071 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":134,"threads_started":1}
I20260811 14:30:21.107290 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.102813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53870 (local address 127.23.186.193:43613)
0811 14:30:21.103175 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.103179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.103187 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:21.103220 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.103221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.103233 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.103323 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:21.103500 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.104368 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.105908 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.106672 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.107060 (+ 388us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.107137 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.107163 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.107210 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":95,"threads_started":1}
I20260811 14:30:21.107328 25508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.102704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:44949 (local address 127.0.0.1:39806)
0811 14:30:21.102985 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:21.102994 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:21.103037 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:21.103716 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:21.103717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:21.103719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:21.103764 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.103765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.104988 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.104989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.105849 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.105851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.106832 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.106833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.106889 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.106891 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:21.106935 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 14:30:21.106966 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260811 14:30:21.107019 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.102703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:45763 (local address 127.0.0.1:34208)
0811 14:30:21.103284 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:21.103296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:21.103352 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:21.103548 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:21.103549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:21.103551 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:21.103585 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.103586 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.104942 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.104943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.105988 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.105990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.106676 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.106677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.106762 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.106777 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 14:30:21.106852 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:30:21.106915 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":434,"thread_start_us":95,"threads_started":1}
I20260811 14:30:21.108125 25419 tablet_service.cc:1467] Processing CreateTablet for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c (DEFAULT_TABLE table=test-table [id=3eea91a0460a4ee4a596560009848071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:21.108177 25103 tablet_service.cc:1467] Processing CreateTablet for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c (DEFAULT_TABLE table=test-table [id=3eea91a0460a4ee4a596560009848071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:21.108388 25264 tablet_service.cc:1467] Processing CreateTablet for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c (DEFAULT_TABLE table=test-table [id=3eea91a0460a4ee4a596560009848071]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:21.108530 25419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:21.108530 25103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:21.108796 25264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:21.111398 25514 tablet_bootstrap.cc:492] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: Bootstrap starting.
I20260811 14:30:21.112011 25516 tablet_bootstrap.cc:492] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7: Bootstrap starting.
I20260811 14:30:21.112274 25514 tablet_bootstrap.cc:654] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:21.112784 25516 tablet_bootstrap.cc:654] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:21.112887 25517 tablet_bootstrap.cc:492] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068: Bootstrap starting.
I20260811 14:30:21.113005 25514 log.cc:824] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:21.113412 25516 log.cc:824] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:21.113556 25517 tablet_bootstrap.cc:654] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:21.113950 25514 tablet_bootstrap.cc:492] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: No bootstrap required, opened a new log
I20260811 14:30:21.114051 25514 ts_tablet_manager.cc:1397] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:30:21.114166 25517 log.cc:824] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:21.114244 25516 tablet_bootstrap.cc:492] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7: No bootstrap required, opened a new log
I20260811 14:30:21.114311 25516 ts_tablet_manager.cc:1397] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:30:21.114984 25517 tablet_bootstrap.cc:492] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068: No bootstrap required, opened a new log
I20260811 14:30:21.115061 25517 ts_tablet_manager.cc:1397] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:30:21.115835 25514 raft_consensus.cc:348] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.116031 25514 raft_consensus.cc:374] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:21.116080 25514 raft_consensus.cc:729] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14630222ef5f41918541f29da92d0a3f, State: Initialized, Role: FOLLOWER
I20260811 14:30:21.116197 25514 consensus_queue.cc:260] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.116423 25516 raft_consensus.cc:348] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.116550 25514 ts_tablet_manager.cc:1428] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:30:21.116559 25341 heartbeater.cc:503] Master 127.23.186.254:36945 was elected leader, sending a full tablet report...
I20260811 14:30:21.116602 25516 raft_consensus.cc:374] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:21.116641 25516 raft_consensus.cc:729] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0d0377b46194e3da4bf4c756d87d7a7, State: Initialized, Role: FOLLOWER
I20260811 14:30:21.116781 25516 consensus_queue.cc:260] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.117094 25189 heartbeater.cc:503] Master 127.23.186.254:36945 was elected leader, sending a full tablet report...
I20260811 14:30:21.117172 25517 raft_consensus.cc:348] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.117336 25517 raft_consensus.cc:374] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:21.117381 25517 raft_consensus.cc:729] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc6c99ddaa174744a19c7897b8ff1068, State: Initialized, Role: FOLLOWER
I20260811 14:30:21.117498 25517 consensus_queue.cc:260] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.117069 25516 ts_tablet_manager.cc:1428] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:21.117797 25517 ts_tablet_manager.cc:1428] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:21.118021 25494 heartbeater.cc:503] Master 127.23.186.254:36945 was elected leader, sending a full tablet report...
I20260811 14:30:21.124822 25522 raft_consensus.cc:482] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:21.124979 25522 raft_consensus.cc:504] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.125409 25522 leader_election.cc:288] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14630222ef5f41918541f29da92d0a3f (127.23.186.194:45763), cc6c99ddaa174744a19c7897b8ff1068 (127.23.186.195:44949)
I20260811 14:30:21.128415 25511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.125807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:51853 (local address 127.23.186.194:45763)
0811 14:30:21.125863 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.125865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.125903 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:30:21.125970 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.125971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.125981 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.126041 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:21.126222 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.126715 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.127688 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.128037 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.128121 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.128132 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.128291 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.128336 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:21.128446 25191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.125561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:44949 (local address 127.23.186.193:44469)
0811 14:30:21.125669 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:21.125684 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:21.125740 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:21.125954 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:21.125956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:21.125958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:21.126060 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.126062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.126721 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.126723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.127565 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.127568 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.128140 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.128141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.128235 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.128238 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:21.128316 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 14:30:21.128355 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82,"mutex_wait_us":39}
I20260811 14:30:21.128444 25512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.125645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:44469 (local address 127.23.186.195:44949)
0811 14:30:21.125694 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.125697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.125731 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:30:21.125862 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.125863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.125876 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.125935 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:21.126137 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.126636 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.127660 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.128038 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.128129 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.128138 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.128336 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.128381 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:30:21.128453 25524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.125589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:45763 (local address 127.23.186.193:51853)
0811 14:30:21.125846 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:21.125857 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:21.125918 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:21.126057 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:21.126058 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:21.126059 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:21.126148 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.126151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.126800 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.126801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.127568 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:21.127569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.128131 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.128132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:21.128208 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.128214 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:21.128259 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 14:30:21.128311 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"mutex_wait_us":22,"thread_start_us":131,"threads_started":1}
I20260811 14:30:21.128821 25443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4c5f662fc4eb5b62163ce5a4cdb0c" candidate_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc6c99ddaa174744a19c7897b8ff1068" is_pre_election: true
I20260811 14:30:21.128988 25443 raft_consensus.cc:2454] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0d0377b46194e3da4bf4c756d87d7a7 in term 0.
I20260811 14:30:21.129263 25051 leader_election.cc:302] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0d0377b46194e3da4bf4c756d87d7a7, cc6c99ddaa174744a19c7897b8ff1068; no voters:
I20260811 14:30:21.129407 25522 raft_consensus.cc:2781] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:21.129400 25287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4c5f662fc4eb5b62163ce5a4cdb0c" candidate_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14630222ef5f41918541f29da92d0a3f" is_pre_election: true
I20260811 14:30:21.129467 25522 raft_consensus.cc:482] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:21.129503 25522 raft_consensus.cc:3037] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:21.129540 25287 raft_consensus.cc:2454] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0d0377b46194e3da4bf4c756d87d7a7 in term 0.
I20260811 14:30:21.130312 25522 raft_consensus.cc:504] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.130477 25522 leader_election.cc:288] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [CANDIDATE]: Term 1 election: Requested vote from peers 14630222ef5f41918541f29da92d0a3f (127.23.186.194:45763), cc6c99ddaa174744a19c7897b8ff1068 (127.23.186.195:44949)
I20260811 14:30:21.130656 25443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4c5f662fc4eb5b62163ce5a4cdb0c" candidate_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc6c99ddaa174744a19c7897b8ff1068"
I20260811 14:30:21.130649 25287 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cce4c5f662fc4eb5b62163ce5a4cdb0c" candidate_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14630222ef5f41918541f29da92d0a3f"
I20260811 14:30:21.130739 25443 raft_consensus.cc:3037] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:21.130739 25287 raft_consensus.cc:3037] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:21.131659 25443 raft_consensus.cc:2454] T cce4c5f662fc4eb5b62163ce5a4cdb0c P cc6c99ddaa174744a19c7897b8ff1068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0d0377b46194e3da4bf4c756d87d7a7 in term 1.
I20260811 14:30:21.131659 25287 raft_consensus.cc:2454] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0d0377b46194e3da4bf4c756d87d7a7 in term 1.
I20260811 14:30:21.131819 25051 leader_election.cc:302] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0d0377b46194e3da4bf4c756d87d7a7, cc6c99ddaa174744a19c7897b8ff1068; no voters:
I20260811 14:30:21.131914 25522 raft_consensus.cc:2781] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:21.132069 25522 raft_consensus.cc:686] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [term 1 LEADER]: Becoming Leader. State: Replica: b0d0377b46194e3da4bf4c756d87d7a7, State: Running, Role: LEADER
I20260811 14:30:21.132169 25522 consensus_queue.cc:237] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } }
I20260811 14:30:21.133106 24957 catalog_manager.cc:5495] T cce4c5f662fc4eb5b62163ce5a4cdb0c P b0d0377b46194e3da4bf4c756d87d7a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to b0d0377b46194e3da4bf4c756d87d7a7 (127.23.186.193). New cstate: current_term: 1 leader_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0d0377b46194e3da4bf4c756d87d7a7" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "14630222ef5f41918541f29da92d0a3f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc6c99ddaa174744a19c7897b8ff1068" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44949 } health_report { overall_health: UNKNOWN } } }
W20260811 14:30:21.140542 25342 tablet.cc:2369] T cce4c5f662fc4eb5b62163ce5a4cdb0c P 14630222ef5f41918541f29da92d0a3f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:21.146618 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.138209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.23.186.193:43613)
0811 14:30:21.138291 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.138293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.138339 (+ 46us) server_negotiation.cc:407] Connection header received
0811 14:30:21.138383 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.138384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.138394 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.138441 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:21.139001 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.139482 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.140182 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.140388 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.140459 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.143336 (+ 2877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:21.143349 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:21.143349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:21.143357 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:21.145303 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.145711 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.145714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.145714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.145784 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.145966 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.145966 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.145967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.146123 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:21.146240 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.146390 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.146481 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:21.173728 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.165848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53886 (local address 127.23.186.193:43613)
0811 14:30:21.165924 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.165926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.166007 (+ 81us) server_negotiation.cc:407] Connection header received
0811 14:30:21.166080 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.166081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.166091 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.166143 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:21.166827 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.167369 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.168161 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.168386 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.168446 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.170970 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:21.170971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:21.170972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:21.170980 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:21.172456 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.172960 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.172962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.172962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.173019 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.173197 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.173197 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.173198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.173293 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:21.173366 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.173563 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.173631 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[24909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:21.200881 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:21.193358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53902 (local address 127.23.186.193:43613)
0811 14:30:21.193450 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:21.193453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:21.193534 (+ 81us) server_negotiation.cc:407] Connection header received
0811 14:30:21.193621 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:21.193622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:21.193633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:21.193678 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:21.194265 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.194753 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.195564 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:21.195754 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:21.195806 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:21.198366 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:21.198368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:21.198368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:21.198376 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:21.199836 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.200173 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.200174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.200174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.200228 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:21.200395 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:21.200396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:21.200396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:21.200498 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:21.200575 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:21.200737 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 14:30:21.200811 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 14:30:21.201480 25080 tablet_service.cc:2908] Rejecting scan request for tablet cce4c5f662fc4eb5b62163ce5a4cdb0c: Uninitialized: clean time has not yet been initialized
I20260811 14:30:21.202800 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25032
I20260811 14:30:21.212625 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25194
I20260811 14:30:21.219798 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25347
I20260811 14:30:21.228426 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 24922
2026-08-11T14:30:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2164 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[25539](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[25539](info): set up 2 sockets
Aug 11 14:30:21 dist-test-slave-cbsf krb5kdc[25539](info): commencing operation
krb5kdc: starting...
W20260811 14:30:22.440304 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:22 dist-test-slave-cbsf krb5kdc[25539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:22.576186 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:34935
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:34935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:22.662959 25554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:22.663261 25554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:22.663304 25554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:22.664536 25554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:22.664624 25554 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 14:30:22.664651 25554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:22.664676 25554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:22.664749 25554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:22.664793 25554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:22.666419 25554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:34935
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:34935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:22.666742 25554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:22.667009 25554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:22.672379 25560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:22.672402 25559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:22.672405 25562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:22.672547 25554 server_base.cc:1034] running on GCE node
I20260811 14:30:22.672851 25554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:22.673189 25554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:22.674340 25554 hybrid_clock.cc:648] HybridClock initialized: now 1786458622674321 us; error 40 us; skew 500 ppm
Aug 11 14:30:22 dist-test-slave-cbsf krb5kdc[25539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458622, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:22.677111 25554 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:22.677656 25554 webserver.cc:460] Webserver started at http://127.23.186.254:39043/ using document root <none> and password file <none>
I20260811 14:30:22.677913 25554 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:22.677964 25554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:22.678112 25554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:22.679250 25554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "8119cb3c6c304594b89e301170fedd6e"
format_stamp: "Formatted at 2026-08-11 14:30:22 on dist-test-slave-cbsf"
server_key: "3f3ea7232c5846c245672abc74411ff6"
server_key_iv: "61d6bfe1568362251a03733e0935c022"
server_key_version: "encryptionkey@0"
I20260811 14:30:22.679646 25554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "8119cb3c6c304594b89e301170fedd6e"
format_stamp: "Formatted at 2026-08-11 14:30:22 on dist-test-slave-cbsf"
server_key: "3f3ea7232c5846c245672abc74411ff6"
server_key_iv: "61d6bfe1568362251a03733e0935c022"
server_key_version: "encryptionkey@0"
I20260811 14:30:22.681192 25554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:22.682029 25569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:22.682265 25554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:22.682370 25554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "8119cb3c6c304594b89e301170fedd6e"
format_stamp: "Formatted at 2026-08-11 14:30:22 on dist-test-slave-cbsf"
server_key: "3f3ea7232c5846c245672abc74411ff6"
server_key_iv: "61d6bfe1568362251a03733e0935c022"
server_key_version: "encryptionkey@0"
I20260811 14:30:22.682461 25554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:22.695130 25554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:22.697790 25554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:22.697963 25554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:22.703181 25554 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:34935
I20260811 14:30:22.703236 25638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:34935 every 8 connection(s)
I20260811 14:30:22.703604 25554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:22.704259 25639 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:22.706712 25639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: Bootstrap starting.
I20260811 14:30:22.707342 25639 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:22.707657 25639 log.cc:824] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:22.708441 25639 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: No bootstrap required, opened a new log
I20260811 14:30:22.711292 25639 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119cb3c6c304594b89e301170fedd6e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 34935 } }
I20260811 14:30:22.711476 25639 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:22.711527 25639 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8119cb3c6c304594b89e301170fedd6e, State: Initialized, Role: FOLLOWER
I20260811 14:30:22.711694 25639 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119cb3c6c304594b89e301170fedd6e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 34935 } }
I20260811 14:30:22.711786 25639 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:22.711833 25639 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:22.711884 25639 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:22.712548 25639 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119cb3c6c304594b89e301170fedd6e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 34935 } }
I20260811 14:30:22.712656 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25554
I20260811 14:30:22.712716 25639 leader_election.cc:302] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8119cb3c6c304594b89e301170fedd6e; no voters:
I20260811 14:30:22.712771 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:22.712924 25639 leader_election.cc:288] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:22.712960 24299 external_mini_cluster.cc:1428] Setting key 15148d0906726ce86f4d00965e6b35dc
I20260811 14:30:22.712973 25642 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:22.713156 25642 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [term 1 LEADER]: Becoming Leader. State: Replica: 8119cb3c6c304594b89e301170fedd6e, State: Running, Role: LEADER
I20260811 14:30:22.713225 25639 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:22.713336 25642 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119cb3c6c304594b89e301170fedd6e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 34935 } }
I20260811 14:30:22.713755 25644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8119cb3c6c304594b89e301170fedd6e. Latest consensus state: current_term: 1 leader_uuid: "8119cb3c6c304594b89e301170fedd6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119cb3c6c304594b89e301170fedd6e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 34935 } } }
I20260811 14:30:22.713840 25644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:22.714013 25643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8119cb3c6c304594b89e301170fedd6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119cb3c6c304594b89e301170fedd6e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 34935 } } }
I20260811 14:30:22.714068 25643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:22.714874 25652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:22.715440 25652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:22.717185 25652 catalog_manager.cc:1347] Generated new cluster ID: 731e57547c38471193e53c2e03c23c5a
I20260811 14:30:22.717264 25652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:30:22 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:22.725970 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:22.713636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.23.186.254:34935)
0811 14:30:22.713796 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:22.713807 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:22.713929 (+ 122us) server_negotiation.cc:407] Connection header received
0811 14:30:22.714074 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:22.714084 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:22.714153 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:22.714323 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:22.715194 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.716683 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.717808 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.718423 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.718527 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:22.721582 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:22.721587 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:22.721587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:22.721601 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:22.723942 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:22.724404 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:22.724408 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:22.724408 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:22.724550 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:22.724848 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:22.724850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:22.724850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:22.725221 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:22.725403 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:22.725564 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 14:30:22.725725 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260811 14:30:22.726367 25652 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:22.727109 25652 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:22.735065 25652 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: Generated new TSK 0
I20260811 14:30:22.735309 25652 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:22.768751 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:34935
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:22.859491 25669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:22.859769 25669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:22.859814 25669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:22.861114 25669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:22.861197 25669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:22.861232 25669 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:22.861270 25669 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:22.862857 25669 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:34935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:22.863161 25669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:22.863422 25669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:22.869033 25674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:22.869037 25675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:22.869323 25677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:22.869489 25669 server_base.cc:1034] running on GCE node
I20260811 14:30:22.869765 25669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:22.870039 25669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:22.871261 25669 hybrid_clock.cc:648] HybridClock initialized: now 1786458622871210 us; error 63 us; skew 500 ppm
Aug 11 14:30:22 dist-test-slave-cbsf krb5kdc[25539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458622, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:22.874189 25669 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:22.874722 25669 webserver.cc:460] Webserver started at http://127.23.186.193:33605/ using document root <none> and password file <none>
I20260811 14:30:22.874926 25669 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:22.874971 25669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:22.875068 25669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:22.876536 25669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "24bc20ae10404eb4ac46735a6ceba53d"
format_stamp: "Formatted at 2026-08-11 14:30:22 on dist-test-slave-cbsf"
server_key: "4f9aa01bbde24242295cf31b9044efb5"
server_key_iv: "f9a46feacc56340fd77e18cd8adfcef0"
server_key_version: "encryptionkey@0"
I20260811 14:30:22.876969 25669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "24bc20ae10404eb4ac46735a6ceba53d"
format_stamp: "Formatted at 2026-08-11 14:30:22 on dist-test-slave-cbsf"
server_key: "4f9aa01bbde24242295cf31b9044efb5"
server_key_iv: "f9a46feacc56340fd77e18cd8adfcef0"
server_key_version: "encryptionkey@0"
I20260811 14:30:22.878533 25669 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:22.879349 25685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:22.879588 25669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:22.879699 25669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "24bc20ae10404eb4ac46735a6ceba53d"
format_stamp: "Formatted at 2026-08-11 14:30:22 on dist-test-slave-cbsf"
server_key: "4f9aa01bbde24242295cf31b9044efb5"
server_key_iv: "f9a46feacc56340fd77e18cd8adfcef0"
server_key_version: "encryptionkey@0"
I20260811 14:30:22.879786 25669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:22.904588 25669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:22.907099 25669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:22.907279 25669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:22.908015 25669 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:22.908070 25669 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:22.908110 25669 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:22.908124 25669 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:30:22 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458622, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:22.916004 25669 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:38519
I20260811 14:30:22.916167 25815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:38519 every 8 connection(s)
I20260811 14:30:22.916445 25669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:22.918746 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:22.908405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:39883 (local address 127.23.186.254:34935)
0811 14:30:22.908477 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:22.908480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:22.908784 (+ 304us) server_negotiation.cc:407] Connection header received
0811 14:30:22.909304 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:22.909306 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:22.909326 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:22.909410 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:22.910118 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.910643 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.911690 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.911908 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.912006 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:22.915815 (+ 3809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:22.915817 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:22.915817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:22.915826 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:22.917527 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:22.917918 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:22.917919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:22.917919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:22.917977 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:22.918199 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:22.918199 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:22.918200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:22.918315 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:22.918367 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:22.918601 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 14:30:22.918679 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:30:22.918902 25694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:22.908401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.23.186.193:39883)
0811 14:30:22.908596 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:22.908641 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:22.909199 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:22.909536 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:22.909541 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:22.909799 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:22.910002 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:22.910006 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.910747 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.910748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:22.911585 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:22.911588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.912020 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.912021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:22.912170 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:22.912825 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:22.912838 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:22.915692 (+ 2854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:22.917606 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:22.917611 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:22.917614 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:22.917839 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:22.918045 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:22.918046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:22.918046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:22.918144 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:22.918356 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:22.918358 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:22.918490 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 14:30:22.918571 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:30:22.918702 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":113,"threads_started":1}
I20260811 14:30:22.919539 25816 heartbeater.cc:347] Connected to a master server at 127.23.186.254:34935
I20260811 14:30:22.919624 25816 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:22.919842 25816 heartbeater.cc:511] Master 127.23.186.254:34935 requested a full tablet report, sending...
I20260811 14:30:22.920428 25590 ts_manager.cc:194] Registered new tserver with Master: 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193:38519)
I20260811 14:30:22.921247 25590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:39883
I20260811 14:30:22.926345 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25669
I20260811 14:30:22.926443 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:22.926632 24299 external_mini_cluster.cc:1428] Setting key 65b08a3197c868680376d931ba6ec59f
I20260811 14:30:22.928145 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:22.920833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:40249 (local address 127.23.186.254:34935)
0811 14:30:22.920876 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:22.920878 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:22.921023 (+ 145us) server_negotiation.cc:407] Connection header received
0811 14:30:22.921093 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:22.921093 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:22.921104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:22.921154 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:22.921585 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.922261 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.923655 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.923949 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.924048 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:22.925272 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:22.925274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:22.925274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:22.925282 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:22.927030 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:22.927413 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:22.927414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:22.927414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:22.927480 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:22.927696 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:22.927697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:22.927697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:22.927796 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:22.927842 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:22.928008 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 14:30:22.928080 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:30:22.928287 25823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:22.920784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.23.186.193:40249)
0811 14:30:22.920951 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:22.920961 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:22.921040 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:22.921157 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:22.921158 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:22.921405 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:22.921498 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:22.921502 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.922390 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.922391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:22.923555 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:22.923558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:22.924049 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:22.924051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:22.924138 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:22.924706 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:22.924713 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:22.925172 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:22.927091 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:22.927092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:22.927092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:22.927323 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:22.927553 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:22.927554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:22.927555 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:22.927631 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:22.927856 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:22.927857 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:22.927913 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 14:30:22.927992 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:30:22.928128 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:22.957228 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:34935
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:23.056000 25826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:23.056233 25826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:23.056258 25826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:23.057561 25826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:23.057611 25826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:23.057633 25826 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:23.057651 25826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:23.059121 25826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:34935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:23.059338 25826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:23.059543 25826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:23.064841 25832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:23.064870 25826 server_base.cc:1034] running on GCE node
W20260811 14:30:23.064846 25831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:23.064841 25834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:23.065353 25826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:23.065626 25826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:23.066816 25826 hybrid_clock.cc:648] HybridClock initialized: now 1786458623066798 us; error 31 us; skew 500 ppm
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:23.069669 25826 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:23.070161 25826 webserver.cc:460] Webserver started at http://127.23.186.194:42681/ using document root <none> and password file <none>
I20260811 14:30:23.070407 25826 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:23.070487 25826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:23.070644 25826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:23.071830 25826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff"
format_stamp: "Formatted at 2026-08-11 14:30:23 on dist-test-slave-cbsf"
server_key: "7a6adb1b6fedc66dd649830498f3a302"
server_key_iv: "5989a93ce38e00950331f488d1cdd9a5"
server_key_version: "encryptionkey@0"
I20260811 14:30:23.072232 25826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff"
format_stamp: "Formatted at 2026-08-11 14:30:23 on dist-test-slave-cbsf"
server_key: "7a6adb1b6fedc66dd649830498f3a302"
server_key_iv: "5989a93ce38e00950331f488d1cdd9a5"
server_key_version: "encryptionkey@0"
I20260811 14:30:23.073817 25826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:23.074604 25841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.074800 25826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:23.074913 25826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff"
format_stamp: "Formatted at 2026-08-11 14:30:23 on dist-test-slave-cbsf"
server_key: "7a6adb1b6fedc66dd649830498f3a302"
server_key_iv: "5989a93ce38e00950331f488d1cdd9a5"
server_key_version: "encryptionkey@0"
I20260811 14:30:23.074999 25826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:23.086393 25826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:23.088938 25826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:23.089123 25826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:23.089697 25826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:23.089767 25826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.089819 25826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:23.089847 25826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.097424 25826 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:34867
I20260811 14:30:23.097591 25970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:34867 every 8 connection(s)
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:23.097887 25826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:23.101308 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.090295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:59377 (local address 127.23.186.254:34935)
0811 14:30:23.090378 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.090381 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.090574 (+ 193us) server_negotiation.cc:407] Connection header received
0811 14:30:23.091097 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.091099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.091114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.091196 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.092155 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.093053 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.094566 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.094929 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.095022 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.098335 (+ 3313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.098338 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.098338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.098349 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.099936 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.100364 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.100365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.100365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.100418 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.100641 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.100642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.100642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.100844 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.100899 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.101165 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.101242 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:30:23.101492 25850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.090221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.23.186.194:59377)
0811 14:30:23.090434 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.090478 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.090988 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.091322 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.091327 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.091702 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:23.092012 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.092018 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.093163 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.093165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.094443 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.094446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.095073 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.095074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.095244 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.095950 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:23.095959 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:23.098220 (+ 2261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:23.100031 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.100038 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.100040 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.100285 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.100480 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.100480 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.100481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.100585 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.100904 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:23.100909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:23.101044 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.101136 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.101271 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":120,"threads_started":1}
I20260811 14:30:23.102043 25972 heartbeater.cc:347] Connected to a master server at 127.23.186.254:34935
I20260811 14:30:23.102155 25972 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:23.102346 25972 heartbeater.cc:511] Master 127.23.186.254:34935 requested a full tablet report, sending...
I20260811 14:30:23.102816 25589 ts_manager.cc:194] Registered new tserver with Master: 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867)
I20260811 14:30:23.103225 25589 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:59377
I20260811 14:30:23.103991 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25826
I20260811 14:30:23.104101 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:23.104332 24299 external_mini_cluster.cc:1428] Setting key 5040f13145c7ec47fc63a92eb2d98928
I20260811 14:30:23.109869 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.103593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:33213 (local address 127.23.186.254:34935)
0811 14:30:23.103638 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.103640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.103793 (+ 153us) server_negotiation.cc:407] Connection header received
0811 14:30:23.103829 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.103830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.103843 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.103920 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.104284 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.104992 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.105732 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.105941 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.106018 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.107029 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.107030 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.107031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.107038 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.108435 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.108934 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.108936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.108936 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.109004 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.109249 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.109251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.109251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.109416 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.109527 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.109726 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.109800 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":20}
I20260811 14:30:23.109942 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.103528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.23.186.194:33213)
0811 14:30:23.103711 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.103722 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.103784 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.103925 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.103926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.104137 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:23.104199 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.104201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.105094 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.105095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.105630 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.105632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.106016 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.106017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.106096 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.106615 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:23.106619 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:23.106927 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:23.108570 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.108571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.108572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.108852 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.109099 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.109100 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.109101 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.109176 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.109536 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:23.109538 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:23.109601 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.109695 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.109823 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":103,"threads_started":1}
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:23.135375 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:34935
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:23.222200 25981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:23.222426 25981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:23.222453 25981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:23.223685 25981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:23.223731 25981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:23.223752 25981 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:23.223770 25981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:23.225323 25981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:34935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:23.225591 25981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:23.225862 25981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:23.231110 25988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:23.231160 25990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:23.231175 25987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:23.231530 25981 server_base.cc:1034] running on GCE node
I20260811 14:30:23.231802 25981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:23.232087 25981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:23.233263 25981 hybrid_clock.cc:648] HybridClock initialized: now 1786458623233248 us; error 48 us; skew 500 ppm
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:23.236362 25981 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:23.237006 25981 webserver.cc:460] Webserver started at http://127.23.186.195:44121/ using document root <none> and password file <none>
I20260811 14:30:23.237284 25981 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:23.237363 25981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:23.237546 25981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:23.238765 25981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "939185158a47479e8bdec98be0dc018d"
format_stamp: "Formatted at 2026-08-11 14:30:23 on dist-test-slave-cbsf"
server_key: "0fdb1c8f15a4e4c40536117c3bc82103"
server_key_iv: "a04b63ce4bcb45a4a886c02cb8bcfd37"
server_key_version: "encryptionkey@0"
I20260811 14:30:23.239259 25981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "939185158a47479e8bdec98be0dc018d"
format_stamp: "Formatted at 2026-08-11 14:30:23 on dist-test-slave-cbsf"
server_key: "0fdb1c8f15a4e4c40536117c3bc82103"
server_key_iv: "a04b63ce4bcb45a4a886c02cb8bcfd37"
server_key_version: "encryptionkey@0"
I20260811 14:30:23.240839 25981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:30:23.241729 25997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.241954 25981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:23.242081 25981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "939185158a47479e8bdec98be0dc018d"
format_stamp: "Formatted at 2026-08-11 14:30:23 on dist-test-slave-cbsf"
server_key: "0fdb1c8f15a4e4c40536117c3bc82103"
server_key_iv: "a04b63ce4bcb45a4a886c02cb8bcfd37"
server_key_version: "encryptionkey@0"
I20260811 14:30:23.242182 25981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:23.264670 25981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:23.267554 25981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:23.267772 25981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:23.268497 25981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:23.268558 25981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.268600 25981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:23.268651 25981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:23.277145 25981 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:35611
I20260811 14:30:23.277236 26126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:35611 every 8 connection(s)
I20260811 14:30:23.277704 25981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:23.279974 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.269004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:50899 (local address 127.23.186.254:34935)
0811 14:30:23.269088 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.269091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.269389 (+ 298us) server_negotiation.cc:407] Connection header received
0811 14:30:23.270198 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.270199 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.270232 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.270301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.271393 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.272024 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.273245 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.273455 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.273546 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.276572 (+ 3026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.276574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.276574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.276583 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.278526 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.278937 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.278938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.278938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.279002 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.279305 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.279306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.279306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.279455 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.279519 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.279806 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.279904 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 14:30:23.280061 26009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.269014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.23.186.195:50899)
0811 14:30:23.269197 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.269259 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.270066 (+ 807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.270427 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.270433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.270900 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:23.271290 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.271300 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.272137 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.272138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.273141 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.273146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.273560 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.273562 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.273705 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.274333 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:23.274342 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:23.276424 (+ 2082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:23.278613 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.278619 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.278621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.278856 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.279064 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.279065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.279108 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.279232 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.279519 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:23.279525 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:23.279657 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.279747 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.279889 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
I20260811 14:30:23.280716 26128 heartbeater.cc:347] Connected to a master server at 127.23.186.254:34935
I20260811 14:30:23.280828 26128 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:23.281030 26128 heartbeater.cc:511] Master 127.23.186.254:34935 requested a full tablet report, sending...
I20260811 14:30:23.281451 25590 ts_manager.cc:194] Registered new tserver with Master: 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611)
I20260811 14:30:23.281966 25590 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:50899
I20260811 14:30:23.282301 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 25981
I20260811 14:30:23.282435 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:23.282665 24299 external_mini_cluster.cc:1428] Setting key 25f136a53f8eceee2f1c3b5611e20b29
I20260811 14:30:23.283772 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:30:23.289579 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.282113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53107 (local address 127.23.186.254:34935)
0811 14:30:23.282428 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.282431 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.282555 (+ 124us) server_negotiation.cc:407] Connection header received
0811 14:30:23.282689 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.282690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.282702 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.282759 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.283263 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.283973 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.284807 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.285139 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.285224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.286497 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.286499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.286499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.286508 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.288199 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.288616 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.288617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.288618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.288687 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.288970 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.288971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.288971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.289107 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.289189 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.289373 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.289468 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285}
I20260811 14:30:23.289588 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.282017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.23.186.195:53107)
0811 14:30:23.282494 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.282506 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.282572 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.282836 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.282837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.283072 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:23.283169 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.283172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.284087 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.284088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.284676 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.284677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.285237 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.285238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.285323 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.285928 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:23.285934 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:23.286382 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:23.288321 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.288323 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.288324 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.288529 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.288815 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.288816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.288816 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.288890 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.289190 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:23.289192 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:23.289247 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.289342 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.289471 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":82,"threads_started":1}
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:23.311908 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.303816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.23.186.254:34935)
0811 14:30:23.303889 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.303892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.303903 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:23.303971 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.303972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.303982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.304046 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.304782 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.305362 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.306197 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.306420 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.306522 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.309188 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.309190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.309190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.309198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.310573 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.310989 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.310991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.310991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.311051 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.311274 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.311275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.311275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.311414 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.311544 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.311694 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.311789 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:30:23.313997 25590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:23.315690 25590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:23.322880 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.318397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:34867 (local address 127.0.0.1:35680)
0811 14:30:23.318755 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.318766 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.318830 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.319388 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.319389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.319390 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.319425 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.319426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.320771 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.320773 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.321828 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.321831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.322558 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.322643 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.322657 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.322730 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.322787 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":20,"thread_start_us":94,"threads_started":1}
I20260811 14:30:23.322896 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.318632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.23.186.194:34867)
0811 14:30:23.318923 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.318928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.318937 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:23.318979 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.318981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.318994 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.319324 (+ 330us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.319498 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.320645 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.321921 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.322471 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322562 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.322623 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.322752 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.322791 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":82,"threads_started":1}
I20260811 14:30:23.323014 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.318621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38519 (local address 127.0.0.1:43274)
0811 14:30:23.318764 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.318772 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.318811 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.319320 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.319321 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.319323 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.319385 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.319386 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.320376 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.320377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.321685 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.321688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.322739 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.322820 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.322825 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.322872 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.322919 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":89,"threads_started":1}
I20260811 14:30:23.323127 26148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.318397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:35611 (local address 127.0.0.1:46382)
0811 14:30:23.318591 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.318633 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.318699 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.319244 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.319249 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.319251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.319329 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.319331 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.320766 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.320767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.321935 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.321937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.322900 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322900 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.322951 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.322954 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.323013 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.323054 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":45,"thread_start_us":71,"threads_started":1}
I20260811 14:30:23.323385 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.318911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43274 (local address 127.23.186.193:38519)
0811 14:30:23.319118 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.319122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.319131 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:23.319201 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.319204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.319219 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.319460 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.319522 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.320229 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.321783 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.322566 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322645 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.322700 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.323200 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.323253 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":118,"threads_started":1}
I20260811 14:30:23.323421 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.318492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46382 (local address 127.23.186.195:35611)
0811 14:30:23.318934 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.318940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.318951 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:23.318996 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.318998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.319013 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.319090 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.319418 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.320648 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322031 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.322750 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.322818 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.322867 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.323269 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.323318 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":120,"threads_started":1}
I20260811 14:30:23.324083 25741 tablet_service.cc:1467] Processing CreateTablet for tablet 869e7e090cb64052acd52f49f6e38e69 (DEFAULT_TABLE table=test-table [id=d93da978e6e34a33a80cf1e54efe42a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:23.324088 25896 tablet_service.cc:1467] Processing CreateTablet for tablet 869e7e090cb64052acd52f49f6e38e69 (DEFAULT_TABLE table=test-table [id=d93da978e6e34a33a80cf1e54efe42a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:23.324221 26054 tablet_service.cc:1467] Processing CreateTablet for tablet 869e7e090cb64052acd52f49f6e38e69 (DEFAULT_TABLE table=test-table [id=d93da978e6e34a33a80cf1e54efe42a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:23.324501 25741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 869e7e090cb64052acd52f49f6e38e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.324507 25896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 869e7e090cb64052acd52f49f6e38e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.324540 26054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 869e7e090cb64052acd52f49f6e38e69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.327589 26155 tablet_bootstrap.cc:492] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Bootstrap starting.
I20260811 14:30:23.328392 26157 tablet_bootstrap.cc:492] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Bootstrap starting.
I20260811 14:30:23.328449 26155 tablet_bootstrap.cc:654] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.328589 26156 tablet_bootstrap.cc:492] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Bootstrap starting.
I20260811 14:30:23.328943 26155 log.cc:824] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:23.329138 26157 tablet_bootstrap.cc:654] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.329435 26156 tablet_bootstrap.cc:654] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.329444 26157 log.cc:824] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:23.329752 26156 log.cc:824] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:23.330295 26155 tablet_bootstrap.cc:492] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: No bootstrap required, opened a new log
I20260811 14:30:23.330355 26155 ts_tablet_manager.cc:1397] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260811 14:30:23.330510 26156 tablet_bootstrap.cc:492] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: No bootstrap required, opened a new log
I20260811 14:30:23.330590 26156 ts_tablet_manager.cc:1397] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:23.331938 26155 raft_consensus.cc:348] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.332116 26155 raft_consensus.cc:374] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.332141 26155 raft_consensus.cc:729] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24bc20ae10404eb4ac46735a6ceba53d, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.332283 26155 consensus_queue.cc:260] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.332384 26156 raft_consensus.cc:348] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.332568 26156 raft_consensus.cc:374] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.332619 26156 raft_consensus.cc:729] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939185158a47479e8bdec98be0dc018d, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.332628 25816 heartbeater.cc:503] Master 127.23.186.254:34935 was elected leader, sending a full tablet report...
I20260811 14:30:23.332792 26156 consensus_queue.cc:260] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.332952 26155 ts_tablet_manager.cc:1428] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:30:23.333069 26156 ts_tablet_manager.cc:1428] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:30:23.333180 26157 tablet_bootstrap.cc:492] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: No bootstrap required, opened a new log
I20260811 14:30:23.333187 26128 heartbeater.cc:503] Master 127.23.186.254:34935 was elected leader, sending a full tablet report...
I20260811 14:30:23.333247 26157 ts_tablet_manager.cc:1397] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 14:30:23.335001 26157 raft_consensus.cc:348] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.335119 26157 raft_consensus.cc:374] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.335137 26157 raft_consensus.cc:729] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e8e67cc4ad46bfba2af7bdf77f11ff, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.335255 26157 consensus_queue.cc:260] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.335482 26157 ts_tablet_manager.cc:1428] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:30:23.335572 25972 heartbeater.cc:503] Master 127.23.186.254:34935 was elected leader, sending a full tablet report...
W20260811 14:30:23.348562 25973 tablet.cc:2369] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:30:23.417492 25817 tablet.cc:2369] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:30:23.528636 26129 tablet.cc:2369] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:23.588151 26162 raft_consensus.cc:482] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:23.588300 26162 raft_consensus.cc:504] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.588740 26162 leader_election.cc:288] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611), 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867)
I20260811 14:30:23.591899 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.589040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:45541 (local address 127.23.186.194:34867)
0811 14:30:23.589089 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.589092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.589261 (+ 169us) server_negotiation.cc:407] Connection header received
0811 14:30:23.589302 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.589303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.589316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.589371 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.589584 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.590096 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.591155 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.591522 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.591577 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.591584 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.591770 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.591822 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:30:23.591940 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.589073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:35889 (local address 127.23.186.195:35611)
0811 14:30:23.589123 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.589126 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.589365 (+ 239us) server_negotiation.cc:407] Connection header received
0811 14:30:23.589399 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.589400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.589410 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.589473 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.589687 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.590194 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.591193 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.591589 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.591665 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.591675 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.591837 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.591878 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:30:23.591938 26167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.588937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:34867 (local address 127.23.186.193:45541)
0811 14:30:23.589111 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.589123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.589201 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.589396 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.589397 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.589400 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.589499 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.589501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.590191 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.590192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.591058 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.591060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.591611 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.591612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.591687 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.591692 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.591750 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.591787 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260811 14:30:23.591966 26165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.588951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:35611 (local address 127.23.186.193:35889)
0811 14:30:23.589256 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.589270 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.589338 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.589491 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.589492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.589494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.589584 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.589585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.590278 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.590279 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.591062 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.591064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.591673 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.591674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.591751 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.591756 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.591812 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.591849 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":30,"thread_start_us":62,"threads_started":1}
I20260811 14:30:23.592366 25919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869e7e090cb64052acd52f49f6e38e69" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" is_pre_election: true
I20260811 14:30:23.592352 26076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869e7e090cb64052acd52f49f6e38e69" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939185158a47479e8bdec98be0dc018d" is_pre_election: true
I20260811 14:30:23.592563 25919 raft_consensus.cc:2454] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 0.
I20260811 14:30:23.592576 26076 raft_consensus.cc:2454] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 0.
I20260811 14:30:23.592841 25687 leader_election.cc:302] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24bc20ae10404eb4ac46735a6ceba53d, 79e8e67cc4ad46bfba2af7bdf77f11ff; no voters:
I20260811 14:30:23.592995 26162 raft_consensus.cc:2781] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:23.593056 26162 raft_consensus.cc:482] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:23.593087 26162 raft_consensus.cc:3037] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.593842 26162 raft_consensus.cc:504] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.594044 26162 leader_election.cc:288] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 election: Requested vote from peers 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611), 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867)
I20260811 14:30:23.594210 25919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869e7e090cb64052acd52f49f6e38e69" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff"
I20260811 14:30:23.594259 26076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "869e7e090cb64052acd52f49f6e38e69" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939185158a47479e8bdec98be0dc018d"
I20260811 14:30:23.594311 25919 raft_consensus.cc:3037] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.594378 26076 raft_consensus.cc:3037] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.595058 25919 raft_consensus.cc:2454] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 1.
I20260811 14:30:23.595279 26076 raft_consensus.cc:2454] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 1.
I20260811 14:30:23.595291 25687 leader_election.cc:302] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24bc20ae10404eb4ac46735a6ceba53d, 79e8e67cc4ad46bfba2af7bdf77f11ff; no voters:
I20260811 14:30:23.595376 26162 raft_consensus.cc:2781] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:23.595503 26162 raft_consensus.cc:686] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 1 LEADER]: Becoming Leader. State: Replica: 24bc20ae10404eb4ac46735a6ceba53d, State: Running, Role: LEADER
I20260811 14:30:23.595602 26162 consensus_queue.cc:237] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } }
I20260811 14:30:23.596524 25590 catalog_manager.cc:5495] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d reported cstate change: term changed from 0 to 1, leader changed from <none> to 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193). New cstate: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:23.608780 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.606274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.23.186.193:38519)
0811 14:30:23.606343 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.606345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.606355 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:23.606444 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.606445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.606463 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.606524 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:23.606709 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.607203 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.607986 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.608180 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.608231 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.608423 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.608524 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.608584 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.608623 (+ 39us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.608642 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:30:23.610849 25919 raft_consensus.cc:1261] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Refusing update from remote peer 24bc20ae10404eb4ac46735a6ceba53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:23.610953 26076 raft_consensus.cc:1261] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Refusing update from remote peer 24bc20ae10404eb4ac46735a6ceba53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:23.611202 26162 consensus_queue.cc:1035] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:23.611340 26168 consensus_queue.cc:1035] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:23.614305 26172 mvcc.cc:205] Tried to move back new op lower bound from 7317334522306826240 to 7317334522248318976. Current Snapshot: MvccSnapshot[applied={T|T < 7317334522306826240}]
I20260811 14:30:23.614331 26173 mvcc.cc:205] Tried to move back new op lower bound from 7317334522306826240 to 7317334522248318976. Current Snapshot: MvccSnapshot[applied={T|T < 7317334522306826240}]
I20260811 14:30:23.618587 25590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:23.618685 25590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:23.619572 25590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: Sending DeleteTablet for 3 replicas of tablet 869e7e090cb64052acd52f49f6e38e69
I20260811 14:30:23.619961 25896 tablet_service.cc:1514] Processing DeleteTablet for tablet 869e7e090cb64052acd52f49f6e38e69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:23 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:35680
I20260811 14:30:23.619993 26054 tablet_service.cc:1514] Processing DeleteTablet for tablet 869e7e090cb64052acd52f49f6e38e69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:23 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46382
I20260811 14:30:23.620033 25741 tablet_service.cc:1514] Processing DeleteTablet for tablet 869e7e090cb64052acd52f49f6e38e69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:23 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43274
I20260811 14:30:23.620428 26184 tablet_replica.cc:330] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: stopping tablet replica
I20260811 14:30:23.620599 26185 tablet_replica.cc:330] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: stopping tablet replica
I20260811 14:30:23.620597 26184 raft_consensus.cc:2229] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:23.620533 25590 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 14:30:23.620747 26184 raft_consensus.cc:2258] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 14:30:23.620751 25590 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:23.620800 26185 raft_consensus.cc:2229] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:23.620924 26185 raft_consensus.cc:2258] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:23.621111 26186 tablet_replica.cc:330] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: stopping tablet replica
I20260811 14:30:23.621254 26186 raft_consensus.cc:2229] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:23.621570 26185 ts_tablet_manager.cc:1905] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:23.622385 26054 tablet_service.cc:1467] Processing CreateTablet for tablet 6716167932e3473e8e4725269187bf8b (DEFAULT_TABLE table=test-table [id=4f8615bc983a4f4ea260ff24fc83c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:23.622519 26054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6716167932e3473e8e4725269187bf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.622537 25896 tablet_service.cc:1467] Processing CreateTablet for tablet 6716167932e3473e8e4725269187bf8b (DEFAULT_TABLE table=test-table [id=4f8615bc983a4f4ea260ff24fc83c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:23.622660 25896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6716167932e3473e8e4725269187bf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.622428 25741 tablet_service.cc:1467] Processing CreateTablet for tablet 6716167932e3473e8e4725269187bf8b (DEFAULT_TABLE table=test-table [id=4f8615bc983a4f4ea260ff24fc83c580]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:23.623651 25741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6716167932e3473e8e4725269187bf8b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.624069 26156 tablet_bootstrap.cc:492] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Bootstrap starting.
I20260811 14:30:23.624233 26157 tablet_bootstrap.cc:492] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Bootstrap starting.
I20260811 14:30:23.624590 26156 tablet_bootstrap.cc:654] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.624722 26157 tablet_bootstrap.cc:654] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.625238 26155 tablet_bootstrap.cc:492] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Bootstrap starting.
I20260811 14:30:23.625526 26184 ts_tablet_manager.cc:1905] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:23.625679 26155 tablet_bootstrap.cc:654] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.626087 26156 tablet_bootstrap.cc:492] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: No bootstrap required, opened a new log
I20260811 14:30:23.626132 26156 ts_tablet_manager.cc:1397] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:23.626288 26156 raft_consensus.cc:348] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.626360 26156 raft_consensus.cc:374] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.626380 26156 raft_consensus.cc:729] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939185158a47479e8bdec98be0dc018d, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.626420 26156 consensus_queue.cc:260] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.626433 26186 raft_consensus.cc:2258] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:23.626505 26156 ts_tablet_manager.cc:1428] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:23.626612 26185 ts_tablet_manager.cc:1918] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:23.626679 26185 log.cc:1196] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/869e7e090cb64052acd52f49f6e38e69
I20260811 14:30:23.626962 26185 ts_tablet_manager.cc:1939] T 869e7e090cb64052acd52f49f6e38e69 P 939185158a47479e8bdec98be0dc018d: Deleting consensus metadata
I20260811 14:30:23.627390 26157 tablet_bootstrap.cc:492] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: No bootstrap required, opened a new log
I20260811 14:30:23.627403 25574 catalog_manager.cc:4841] TS 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611): tablet 869e7e090cb64052acd52f49f6e38e69 (table test-table [id=d93da978e6e34a33a80cf1e54efe42a7]) successfully deleted
I20260811 14:30:23.627454 26157 ts_tablet_manager.cc:1397] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 14:30:23.627650 26157 raft_consensus.cc:348] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.627723 26157 raft_consensus.cc:374] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.627745 26157 raft_consensus.cc:729] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e8e67cc4ad46bfba2af7bdf77f11ff, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.627753 26184 ts_tablet_manager.cc:1918] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:23.627791 26157 consensus_queue.cc:260] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.627825 26184 log.cc:1196] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/869e7e090cb64052acd52f49f6e38e69
I20260811 14:30:23.627871 26186 ts_tablet_manager.cc:1905] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:23.627969 26157 ts_tablet_manager.cc:1428] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.628799 26184 ts_tablet_manager.cc:1939] T 869e7e090cb64052acd52f49f6e38e69 P 79e8e67cc4ad46bfba2af7bdf77f11ff: Deleting consensus metadata
I20260811 14:30:23.629374 25571 catalog_manager.cc:4841] TS 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867): tablet 869e7e090cb64052acd52f49f6e38e69 (table test-table [id=d93da978e6e34a33a80cf1e54efe42a7]) successfully deleted
I20260811 14:30:23.629667 26186 ts_tablet_manager.cc:1918] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:23.629781 26186 log.cc:1196] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/869e7e090cb64052acd52f49f6e38e69
I20260811 14:30:23.630159 26186 ts_tablet_manager.cc:1939] T 869e7e090cb64052acd52f49f6e38e69 P 24bc20ae10404eb4ac46735a6ceba53d: Deleting consensus metadata
I20260811 14:30:23.630600 25574 catalog_manager.cc:4841] TS 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193:38519): tablet 869e7e090cb64052acd52f49f6e38e69 (table test-table [id=d93da978e6e34a33a80cf1e54efe42a7]) successfully deleted
I20260811 14:30:23.630851 26155 tablet_bootstrap.cc:492] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: No bootstrap required, opened a new log
I20260811 14:30:23.630894 26155 ts_tablet_manager.cc:1397] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260811 14:30:23.631007 26155 raft_consensus.cc:348] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.631067 26155 raft_consensus.cc:374] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.631081 26155 raft_consensus.cc:729] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24bc20ae10404eb4ac46735a6ceba53d, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.631153 26155 consensus_queue.cc:260] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.631253 26155 ts_tablet_manager.cc:1428] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.732669 26164 raft_consensus.cc:482] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:23.732867 26164 raft_consensus.cc:504] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.733222 26164 leader_election.cc:288] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193:38519), 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611)
I20260811 14:30:23.736609 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.733552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:39641 (local address 127.23.186.193:38519)
0811 14:30:23.733609 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.733611 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.733721 (+ 110us) server_negotiation.cc:407] Connection header received
0811 14:30:23.733845 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.733847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.733861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.733894 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.734183 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.734710 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.735818 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.736216 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.736306 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.736318 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.736499 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.736541 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:30:23.736630 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.733456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38519 (local address 127.23.186.194:39641)
0811 14:30:23.733609 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.733622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.733756 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.733976 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.733977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.733979 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.734103 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.734105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.734801 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.734803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.735715 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.735719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.736334 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.736335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.736413 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.736417 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.736481 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.736528 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":2,"thread_start_us":101,"threads_started":1}
I20260811 14:30:23.737195 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6716167932e3473e8e4725269187bf8b" candidate_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24bc20ae10404eb4ac46735a6ceba53d" is_pre_election: true
I20260811 14:30:23.737350 25762 raft_consensus.cc:2454] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e8e67cc4ad46bfba2af7bdf77f11ff in term 0.
I20260811 14:30:23.737552 25845 leader_election.cc:302] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24bc20ae10404eb4ac46735a6ceba53d, 79e8e67cc4ad46bfba2af7bdf77f11ff; no voters:
I20260811 14:30:23.737708 26164 raft_consensus.cc:2781] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:23.737741 26164 raft_consensus.cc:482] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:23.737766 26164 raft_consensus.cc:3037] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.738056 26163 raft_consensus.cc:482] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:23.738127 26163 raft_consensus.cc:504] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.738284 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.733692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:35611 (local address 127.23.186.194:47849)
0811 14:30:23.733915 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.733926 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.733981 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.734825 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.734826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.734828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.734964 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.734966 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.735881 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.735882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.737091 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.737094 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.737849 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.737850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.737962 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.737966 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.738146 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.738188 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":26,"thread_start_us":137,"threads_started":1}
I20260811 14:30:23.738412 26151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.733779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:47849 (local address 127.23.186.195:35611)
0811 14:30:23.733827 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.733829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.734022 (+ 193us) server_negotiation.cc:407] Connection header received
0811 14:30:23.734075 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.734077 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.734088 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.734185 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.735056 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.735724 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.737201 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.737731 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.737803 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.737813 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.738069 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.738119 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:30:23.738459 26163 leader_election.cc:288] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193:38519), 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867)
I20260811 14:30:23.738726 26164 raft_consensus.cc:504] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.738839 26164 leader_election.cc:288] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [CANDIDATE]: Term 1 election: Requested vote from peers 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193:38519), 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611)
I20260811 14:30:23.739097 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6716167932e3473e8e4725269187bf8b" candidate_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24bc20ae10404eb4ac46735a6ceba53d"
I20260811 14:30:23.739132 26076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6716167932e3473e8e4725269187bf8b" candidate_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939185158a47479e8bdec98be0dc018d" is_pre_election: true
I20260811 14:30:23.739183 25762 raft_consensus.cc:3037] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.739214 26076 raft_consensus.cc:2454] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79e8e67cc4ad46bfba2af7bdf77f11ff in term 0.
I20260811 14:30:23.739249 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6716167932e3473e8e4725269187bf8b" candidate_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939185158a47479e8bdec98be0dc018d"
I20260811 14:30:23.739328 26075 raft_consensus.cc:3037] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.740195 26075 raft_consensus.cc:2454] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e8e67cc4ad46bfba2af7bdf77f11ff in term 1.
I20260811 14:30:23.740248 25762 raft_consensus.cc:2454] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79e8e67cc4ad46bfba2af7bdf77f11ff in term 1.
I20260811 14:30:23.740371 25845 leader_election.cc:302] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79e8e67cc4ad46bfba2af7bdf77f11ff, 939185158a47479e8bdec98be0dc018d; no voters:
I20260811 14:30:23.740449 26164 raft_consensus.cc:2781] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:23.740622 26164 raft_consensus.cc:686] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 LEADER]: Becoming Leader. State: Replica: 79e8e67cc4ad46bfba2af7bdf77f11ff, State: Running, Role: LEADER
I20260811 14:30:23.740738 26164 consensus_queue.cc:237] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.741515 25589 catalog_manager.cc:5495] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194). New cstate: current_term: 1 leader_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:23.743659 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.739015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:34097 (local address 127.23.186.193:38519)
0811 14:30:23.739075 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.739077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.739084 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:23.739103 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.739104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.739113 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.739164 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.740888 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.741531 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.742762 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.743287 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.743384 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.743393 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.743560 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.743598 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:30:23.743659 26009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.738673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38519 (local address 127.23.186.195:34097)
0811 14:30:23.738797 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.738809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.738848 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.740500 (+ 1652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.740501 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.740503 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.740564 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.740566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.741625 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.741626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.742662 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.742665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.743383 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.743384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.743463 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.743467 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.743538 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.743589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 14:30:23.743770 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.738939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:41801 (local address 127.23.186.194:34867)
0811 14:30:23.738979 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.738981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.739473 (+ 492us) server_negotiation.cc:407] Connection header received
0811 14:30:23.739559 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.739559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.739568 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.739594 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:23.739776 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.740518 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.742876 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.743334 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.743383 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.743392 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.743671 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.743712 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:30:23.743973 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.738785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:34867 (local address 127.23.186.195:41801)
0811 14:30:23.739417 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.739427 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.739495 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.739643 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.739644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.739646 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:23.739712 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.739714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.740620 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.740621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.742762 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.742765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.743506 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.743507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.743588 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.743591 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.743840 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.743882 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":66,"threads_started":1}
I20260811 14:30:23.744006 25762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6716167932e3473e8e4725269187bf8b" candidate_uuid: "939185158a47479e8bdec98be0dc018d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24bc20ae10404eb4ac46735a6ceba53d" is_pre_election: true
I20260811 14:30:23.744099 25762 raft_consensus.cc:2379] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 939185158a47479e8bdec98be0dc018d in current term 1: Already voted for candidate 79e8e67cc4ad46bfba2af7bdf77f11ff in this term.
I20260811 14:30:23.744431 25919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6716167932e3473e8e4725269187bf8b" candidate_uuid: "939185158a47479e8bdec98be0dc018d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" is_pre_election: true
I20260811 14:30:23.744652 25999 leader_election.cc:302] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 939185158a47479e8bdec98be0dc018d; no voters: 24bc20ae10404eb4ac46735a6ceba53d, 79e8e67cc4ad46bfba2af7bdf77f11ff
I20260811 14:30:23.744858 26163 raft_consensus.cc:2726] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458622, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:23.804885 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.797708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42706 (local address 127.23.186.254:34935)
0811 14:30:23.797795 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.797798 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.797965 (+ 167us) server_negotiation.cc:407] Connection header received
0811 14:30:23.798027 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.798028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.798038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.798081 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.798580 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.799066 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.799808 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.799992 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.800048 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.802351 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.802353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.802353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.802359 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.803750 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.804112 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.804113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.804113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.804163 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.804324 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.804324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.804324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.804419 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.804486 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.804661 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.804818 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:30:23.804994 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.797650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:34935 (local address 127.0.0.1:42706)
0811 14:30:23.797866 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.797880 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.797964 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.798119 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.798120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.798443 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:23.798494 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.798496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.799165 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.799166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.799713 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.799715 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.800068 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.800069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.800116 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.800629 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:23.800633 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:23.802252 (+ 1619us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:23.803825 (+ 1573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.803832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.803834 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.804041 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.804220 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:23.804221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:23.804221 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:23.804277 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:23.804497 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:23.804505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:23.804570 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.804644 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.804904 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"mutex_wait_us":90,"thread_start_us":124,"threads_started":1}
I20260811 14:30:23.807214 25589 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:42706:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 14:30:23.807418 25589 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:23.808991 25896 tablet_service.cc:1467] Processing CreateTablet for tablet bd0ddb89e72a466fb502e00553f9268b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ee11a6145f64d3b92d52bb3716f53b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:23.808984 25741 tablet_service.cc:1467] Processing CreateTablet for tablet bd0ddb89e72a466fb502e00553f9268b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ee11a6145f64d3b92d52bb3716f53b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:23.809049 26054 tablet_service.cc:1467] Processing CreateTablet for tablet bd0ddb89e72a466fb502e00553f9268b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1ee11a6145f64d3b92d52bb3716f53b7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:23.809190 26054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd0ddb89e72a466fb502e00553f9268b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.809190 25896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd0ddb89e72a466fb502e00553f9268b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.809190 25741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd0ddb89e72a466fb502e00553f9268b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:23.810613 26157 tablet_bootstrap.cc:492] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Bootstrap starting.
I20260811 14:30:23.810815 26156 tablet_bootstrap.cc:492] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: Bootstrap starting.
I20260811 14:30:23.811144 26157 tablet_bootstrap.cc:654] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.811233 26156 tablet_bootstrap.cc:654] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.811794 26155 tablet_bootstrap.cc:492] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: Bootstrap starting.
I20260811 14:30:23.812063 26157 tablet_bootstrap.cc:492] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: No bootstrap required, opened a new log
I20260811 14:30:23.812114 26157 ts_tablet_manager.cc:1397] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:23.812235 26155 tablet_bootstrap.cc:654] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:23.812294 26157 raft_consensus.cc:348] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.812402 26157 raft_consensus.cc:374] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.812422 26157 raft_consensus.cc:729] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79e8e67cc4ad46bfba2af7bdf77f11ff, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.812498 26157 consensus_queue.cc:260] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.812628 26157 ts_tablet_manager.cc:1428] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:23.812683 26156 tablet_bootstrap.cc:492] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: No bootstrap required, opened a new log
I20260811 14:30:23.812711 26193 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.812772 26156 ts_tablet_manager.cc:1397] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:23.812810 26193 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.812984 26156 raft_consensus.cc:348] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.813076 26156 raft_consensus.cc:374] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.813102 26156 raft_consensus.cc:729] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 939185158a47479e8bdec98be0dc018d, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.813156 26156 consensus_queue.cc:260] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.813257 26156 ts_tablet_manager.cc:1428] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:23.813315 26163 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.813406 26163 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.813707 26155 tablet_bootstrap.cc:492] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: No bootstrap required, opened a new log
I20260811 14:30:23.813768 26155 ts_tablet_manager.cc:1397] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:30:23.813905 26155 raft_consensus.cc:348] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.813984 26155 raft_consensus.cc:374] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:23.814015 26155 raft_consensus.cc:729] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24bc20ae10404eb4ac46735a6ceba53d, State: Initialized, Role: FOLLOWER
I20260811 14:30:23.814076 26155 consensus_queue.cc:260] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.814188 26155 ts_tablet_manager.cc:1428] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 14:30:23.814249 26177 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.814373 26177 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.875445 26177 raft_consensus.cc:482] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:23.875578 26177 raft_consensus.cc:504] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.875855 26177 leader_election.cc:288] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867), 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611)
I20260811 14:30:23.876144 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0ddb89e72a466fb502e00553f9268b" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939185158a47479e8bdec98be0dc018d" is_pre_election: true
I20260811 14:30:23.876183 25919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0ddb89e72a466fb502e00553f9268b" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" is_pre_election: true
I20260811 14:30:23.876261 26075 raft_consensus.cc:2454] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 0.
I20260811 14:30:23.876295 25919 raft_consensus.cc:2454] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 0.
I20260811 14:30:23.876565 25687 leader_election.cc:302] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 24bc20ae10404eb4ac46735a6ceba53d, 79e8e67cc4ad46bfba2af7bdf77f11ff, 939185158a47479e8bdec98be0dc018d; no voters:
I20260811 14:30:23.876749 26177 raft_consensus.cc:2781] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:23.876796 26177 raft_consensus.cc:482] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:23.876813 26177 raft_consensus.cc:3037] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.877842 26177 raft_consensus.cc:504] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.878027 26177 leader_election.cc:288] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 election: Requested vote from peers 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867), 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611)
I20260811 14:30:23.878239 25919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0ddb89e72a466fb502e00553f9268b" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff"
I20260811 14:30:23.878273 26075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd0ddb89e72a466fb502e00553f9268b" candidate_uuid: "24bc20ae10404eb4ac46735a6ceba53d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "939185158a47479e8bdec98be0dc018d"
I20260811 14:30:23.878350 26075 raft_consensus.cc:3037] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.878352 25919 raft_consensus.cc:3037] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:23.879197 26075 raft_consensus.cc:2454] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 1.
I20260811 14:30:23.879184 25919 raft_consensus.cc:2454] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 24bc20ae10404eb4ac46735a6ceba53d in term 1.
I20260811 14:30:23.879383 25689 leader_election.cc:302] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24bc20ae10404eb4ac46735a6ceba53d, 939185158a47479e8bdec98be0dc018d; no voters:
I20260811 14:30:23.879622 26177 raft_consensus.cc:2781] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:23.879706 26177 raft_consensus.cc:686] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 LEADER]: Becoming Leader. State: Replica: 24bc20ae10404eb4ac46735a6ceba53d, State: Running, Role: LEADER
I20260811 14:30:23.879772 26177 consensus_queue.cc:237] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } }
I20260811 14:30:23.880095 26168 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: TxnStatusTablet state changed. Reason: New leader 24bc20ae10404eb4ac46735a6ceba53d. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.880204 26168 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:23.880404 26205 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 14:30:23.880538 26205 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 14:30:23.880511 25589 catalog_manager.cc:5495] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d reported cstate change: term changed from 0 to 1, leader changed from <none> to 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193). New cstate: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:23.889129 26200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.886082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38519 (local address 127.0.0.1:43286)
0811 14:30:23.886185 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.886197 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.886275 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.886468 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.886469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.886470 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:23.886525 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.886526 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.887492 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.887494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.888108 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.888110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.888622 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.888623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.888676 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.888753 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.888961 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.888967 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.889013 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.889064 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 14:30:23.889134 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.886153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.23.186.193:38519)
0811 14:30:23.886206 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.886209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.886252 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:30:23.886360 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.886361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.886373 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.886462 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:23.886586 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.887374 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.888213 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.888519 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.888597 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.888851 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.888891 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.888948 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.889017 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.889043 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:30:23.890228 25919 raft_consensus.cc:1261] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Refusing update from remote peer 24bc20ae10404eb4ac46735a6ceba53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:23.890337 26075 raft_consensus.cc:1261] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Refusing update from remote peer 24bc20ae10404eb4ac46735a6ceba53d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:23.890542 26177 consensus_queue.cc:1035] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:23.890650 26168 consensus_queue.cc:1035] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d [LEADER]: Connected to new peer: Peer: permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:23.891213 26163 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: TxnStatusTablet state changed. Reason: New leader 24bc20ae10404eb4ac46735a6ceba53d. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.891324 26163 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.891268 26193 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: TxnStatusTablet state changed. Reason: New leader 24bc20ae10404eb4ac46735a6ceba53d. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.891358 26193 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.891906 26168 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.891991 26168 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:23.892242 26163 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.892333 26163 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 939185158a47479e8bdec98be0dc018d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.892975 26177 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.893085 26177 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 24bc20ae10404eb4ac46735a6ceba53d: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:23.893548 26193 tablet_replica.cc:439] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "24bc20ae10404eb4ac46735a6ceba53d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 } } peers { permanent_uuid: "79e8e67cc4ad46bfba2af7bdf77f11ff" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34867 } } peers { permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 } } }
I20260811 14:30:23.893661 26193 tablet_replica.cc:442] T bd0ddb89e72a466fb502e00553f9268b P 79e8e67cc4ad46bfba2af7bdf77f11ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:23.896936 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.894139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35696 (local address 127.23.186.194:34867)
0811 14:30:23.894205 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.894207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.894217 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:23.894287 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.894288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.894312 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.894399 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:23.894478 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.895218 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.895994 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.896308 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.896380 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.896545 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.896687 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.896784 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.896839 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.896859 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:30:23.902781 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.899850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:35121 (local address 127.23.186.193:38519)
0811 14:30:23.899900 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.899902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.900051 (+ 149us) server_negotiation.cc:407] Connection header received
0811 14:30:23.900109 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.900110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.900119 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.900161 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:23.900305 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.900861 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.901966 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.902197 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.902247 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.902510 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.902549 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.902631 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.902697 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.902721 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:30:23.902837 26216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.899749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38519 (local address 127.23.186.194:35121)
0811 14:30:23.899973 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.899985 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.900050 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.900173 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.900174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.900176 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:23.900250 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.900251 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.900936 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.900937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.901854 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.901857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.902269 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.902270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.902356 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.902374 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.902637 (+ 263us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.902639 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.902694 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.902741 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260811 14:30:23.905747 26075 raft_consensus.cc:1261] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Refusing update from remote peer 79e8e67cc4ad46bfba2af7bdf77f11ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:23.905850 25762 raft_consensus.cc:1261] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Refusing update from remote peer 79e8e67cc4ad46bfba2af7bdf77f11ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:23.906103 26193 consensus_queue.cc:1035] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "939185158a47479e8bdec98be0dc018d" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:23.906245 26164 consensus_queue.cc:1035] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "24bc20ae10404eb4ac46735a6ceba53d" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:23.916275 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.913130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:52115 (local address 127.23.186.194:34867)
0811 14:30:23.913214 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.913217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.913420 (+ 203us) server_negotiation.cc:407] Connection header received
0811 14:30:23.913448 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.913449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.913460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.913518 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:23.913669 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.914431 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.915536 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.915735 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.915792 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.915998 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.916047 (+ 49us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.916106 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.916182 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.916208 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 14:30:23.916322 26220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.913096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:34867 (local address 127.23.186.193:52115)
0811 14:30:23.913322 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:23.913336 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:23.913398 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:23.913522 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:23.913522 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:23.913524 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:23.913601 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.913603 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.914560 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.914561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.915447 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:23.915450 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.915820 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.915821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:23.915884 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.915905 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:23.916130 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:23.916132 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:23.916180 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:30:23.916233 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":151,"threads_started":1}
I20260811 14:30:23.930361 25590 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:23.930500 25590 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:23.931232 25590 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8119cb3c6c304594b89e301170fedd6e: Sending DeleteTablet for 3 replicas of tablet 6716167932e3473e8e4725269187bf8b
I20260811 14:30:23.931511 25741 tablet_service.cc:1514] Processing DeleteTablet for tablet 6716167932e3473e8e4725269187bf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:23 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43274
I20260811 14:30:23.931617 26186 tablet_replica.cc:330] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: stopping tablet replica
I20260811 14:30:23.931612 26054 tablet_service.cc:1514] Processing DeleteTablet for tablet 6716167932e3473e8e4725269187bf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:23 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46382
I20260811 14:30:23.931670 26186 raft_consensus.cc:2229] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:23.931710 26186 raft_consensus.cc:2258] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:23.931782 26185 tablet_replica.cc:330] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: stopping tablet replica
I20260811 14:30:23.931840 26185 raft_consensus.cc:2229] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:23.931875 26185 raft_consensus.cc:2258] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:23.931854 25896 tablet_service.cc:1514] Processing DeleteTablet for tablet 6716167932e3473e8e4725269187bf8b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:23 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:35680
I20260811 14:30:23.932026 26184 tablet_replica.cc:330] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: stopping tablet replica
I20260811 14:30:23.932092 26184 raft_consensus.cc:2229] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:23.932185 26184 raft_consensus.cc:2258] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:23.932674 26185 ts_tablet_manager.cc:1905] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:23.932833 26186 ts_tablet_manager.cc:1905] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:23.934481 26184 ts_tablet_manager.cc:1905] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:23.934525 26185 ts_tablet_manager.cc:1918] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:23.934574 26185 log.cc:1196] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/6716167932e3473e8e4725269187bf8b
I20260811 14:30:23.934816 26185 ts_tablet_manager.cc:1939] T 6716167932e3473e8e4725269187bf8b P 939185158a47479e8bdec98be0dc018d: Deleting consensus metadata
I20260811 14:30:23.935251 25574 catalog_manager.cc:4841] TS 939185158a47479e8bdec98be0dc018d (127.23.186.195:35611): tablet 6716167932e3473e8e4725269187bf8b (table test-table [id=4f8615bc983a4f4ea260ff24fc83c580]) successfully deleted
I20260811 14:30:23.936018 26184 ts_tablet_manager.cc:1918] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:23.936071 26184 log.cc:1196] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/6716167932e3473e8e4725269187bf8b
I20260811 14:30:23.936292 26184 ts_tablet_manager.cc:1939] T 6716167932e3473e8e4725269187bf8b P 79e8e67cc4ad46bfba2af7bdf77f11ff: Deleting consensus metadata
I20260811 14:30:23.936621 26186 ts_tablet_manager.cc:1918] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:23.936647 25571 catalog_manager.cc:4841] TS 79e8e67cc4ad46bfba2af7bdf77f11ff (127.23.186.194:34867): tablet 6716167932e3473e8e4725269187bf8b (table test-table [id=4f8615bc983a4f4ea260ff24fc83c580]) successfully deleted
I20260811 14:30:23.936686 26186 log.cc:1196] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/6716167932e3473e8e4725269187bf8b
I20260811 14:30:23.936914 26186 ts_tablet_manager.cc:1939] T 6716167932e3473e8e4725269187bf8b P 24bc20ae10404eb4ac46735a6ceba53d: Deleting consensus metadata
I20260811 14:30:23.937227 25574 catalog_manager.cc:4841] TS 24bc20ae10404eb4ac46735a6ceba53d (127.23.186.193:38519): tablet 6716167932e3473e8e4725269187bf8b (table test-table [id=4f8615bc983a4f4ea260ff24fc83c580]) successfully deleted
Aug 11 14:30:23 dist-test-slave-cbsf krb5kdc[25539](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458623, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:23.951735 26154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.940841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43302 (local address 127.23.186.193:38519)
0811 14:30:23.940891 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.940894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.941079 (+ 185us) server_negotiation.cc:407] Connection header received
0811 14:30:23.941121 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.941123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.941136 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.941198 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.941955 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.942740 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.943935 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.944256 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.944331 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.948146 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.948160 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.948160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.948169 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.950311 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.950740 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.950744 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.950744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.950813 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.951042 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.951043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.951043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.951233 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.951375 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.951483 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.951616 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 14:30:23.952099 25809 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43302
I20260811 14:30:23.964871 25647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:23.953239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42714 (local address 127.23.186.254:34935)
0811 14:30:23.953291 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:23.953293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:23.956949 (+ 3656us) server_negotiation.cc:407] Connection header received
0811 14:30:23.957020 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:23.957022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:23.957038 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:23.957100 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:23.957823 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.958649 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.959688 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:23.960023 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:23.960098 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:23.961616 (+ 1518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:23.961617 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:23.961617 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:23.961626 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:23.963357 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.963925 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.963927 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.963927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.964003 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:23.964223 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:23.964223 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:23.964224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:23.964372 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:23.964478 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:23.964612 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 14:30:23.964749 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 14:30:23.965133 25590 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42714
I20260811 14:30:23.965910 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25669
I20260811 14:30:23.983824 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25826
I20260811 14:30:23.994135 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25981
I20260811 14:30:24.003185 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 25554
2026-08-11T14:30:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2775 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:24 dist-test-slave-cbsf krb5kdc[26230](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:24 dist-test-slave-cbsf krb5kdc[26230](info): set up 2 sockets
Aug 11 14:30:24 dist-test-slave-cbsf krb5kdc[26230](info): commencing operation
krb5kdc: starting...
W20260811 14:30:25.203769 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:25.335978 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41799
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:25.422019 26243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:25.422292 26243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:25.422333 26243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:25.423575 26243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:25.423655 26243 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 14:30:25.423676 26243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:25.423697 26243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:25.423729 26243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:25.423751 26243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:25.425370 26243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:41799
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:25.425695 26243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:25.425940 26243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:25.431316 26250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.431277 26248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.431284 26252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:25.431572 26243 server_base.cc:1034] running on GCE node
I20260811 14:30:25.431854 26243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:25.432128 26243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:25.433305 26243 hybrid_clock.cc:648] HybridClock initialized: now 1786458625433288 us; error 44 us; skew 500 ppm
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:25.435989 26243 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:25.436482 26243 webserver.cc:460] Webserver started at http://127.23.186.254:38861/ using document root <none> and password file <none>
I20260811 14:30:25.436789 26243 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:25.436861 26243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:25.437034 26243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:25.438170 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "4d8c08a8169e497c8543a1c3cdb914d1"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "80be30c65ef6c924a482b88719b6f81e"
server_key_iv: "e4d5521a67821b33766410468a1c1405"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.438557 26243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "4d8c08a8169e497c8543a1c3cdb914d1"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "80be30c65ef6c924a482b88719b6f81e"
server_key_iv: "e4d5521a67821b33766410468a1c1405"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.440179 26243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:25.441318 26259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.441516 26243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:30:25.441633 26243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "4d8c08a8169e497c8543a1c3cdb914d1"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "80be30c65ef6c924a482b88719b6f81e"
server_key_iv: "e4d5521a67821b33766410468a1c1405"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.441727 26243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:25.461951 26243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:25.464136 26243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:25.464319 26243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:25.469358 26243 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:41799
I20260811 14:30:25.469422 26330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:41799 every 8 connection(s)
I20260811 14:30:25.469781 26243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:25.470502 26331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:25.472285 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 26243
I20260811 14:30:25.472394 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:25.472594 24299 external_mini_cluster.cc:1428] Setting key aa941aec74dce30e8ea892ad339cd234
I20260811 14:30:25.473014 26331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Bootstrap starting.
I20260811 14:30:25.473765 26331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:25.474115 26331 log.cc:824] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:25.474941 26331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: No bootstrap required, opened a new log
I20260811 14:30:25.476323 26331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41799 } }
I20260811 14:30:25.476502 26331 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:25.476553 26331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d8c08a8169e497c8543a1c3cdb914d1, State: Initialized, Role: FOLLOWER
I20260811 14:30:25.476730 26331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41799 } }
I20260811 14:30:25.476833 26331 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:25.476878 26331 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:25.476951 26331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:25.477682 26331 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41799 } }
I20260811 14:30:25.477799 26331 leader_election.cc:302] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4d8c08a8169e497c8543a1c3cdb914d1; no voters:
I20260811 14:30:25.477972 26331 leader_election.cc:288] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:25.478180 26337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:25.478297 26331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:25.478325 26337 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [term 1 LEADER]: Becoming Leader. State: Replica: 4d8c08a8169e497c8543a1c3cdb914d1, State: Running, Role: LEADER
I20260811 14:30:25.478507 26337 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41799 } }
I20260811 14:30:25.478894 26336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4d8c08a8169e497c8543a1c3cdb914d1. Latest consensus state: current_term: 1 leader_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41799 } } }
I20260811 14:30:25.478993 26336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:25.479264 26345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:25.479876 26345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:25.479238 26338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4d8c08a8169e497c8543a1c3cdb914d1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41799 } } }
I20260811 14:30:25.480080 26338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1 [sys.catalog]: This master's current role is: LEADER
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:25.482167 26345 catalog_manager.cc:1347] Generated new cluster ID: 29f85e14a63d402492f18e36dbdb6a46
I20260811 14:30:25.482216 26345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:25.485677 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.473227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34672 (local address 127.23.186.254:41799)
0811 14:30:25.473435 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:25.473445 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:25.473485 (+ 40us) server_negotiation.cc:407] Connection header received
0811 14:30:25.474159 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:25.474169 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:25.474232 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:25.474399 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:25.475154 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.476824 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.477969 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.478573 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.478711 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.481871 (+ 3160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:25.481876 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:25.481876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:25.481889 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:25.483876 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.484325 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.484328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.484328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.484430 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.484651 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.484652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.484653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.484978 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:25.485159 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:25.485331 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 14:30:25.485469 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":90,"threads_started":1}
I20260811 14:30:25.488997 26345 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:25.489521 26345 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:25.491758 26345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Generated new TSK 0
I20260811 14:30:25.491946 26345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:25.531518 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41799
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:25.628088 26360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:25.628296 26360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:25.628327 26360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:25.629710 26360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:25.629760 26360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:25.629783 26360 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 14:30:25.629820 26360 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:25.629855 26360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:25.631423 26360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:25.631704 26360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:25.631975 26360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:25.637362 26368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.637378 26365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:25.637584 26360 server_base.cc:1034] running on GCE node
W20260811 14:30:25.637378 26366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:25.637894 26360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:25.638190 26360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:25.639389 26360 hybrid_clock.cc:648] HybridClock initialized: now 1786458625639366 us; error 35 us; skew 500 ppm
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:25.642094 26360 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:25.642580 26360 webserver.cc:460] Webserver started at http://127.23.186.193:34343/ using document root <none> and password file <none>
I20260811 14:30:25.642851 26360 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:25.642925 26360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:25.643082 26360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:25.644250 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "e4f4bb6c4de745f0b184ca7383dc0b08"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "f30f4f8c8a51cba24c026c7eb9e779a3"
server_key_iv: "0e1c56a4c8f98dbafa43ba562202724c"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.644629 26360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "e4f4bb6c4de745f0b184ca7383dc0b08"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "f30f4f8c8a51cba24c026c7eb9e779a3"
server_key_iv: "0e1c56a4c8f98dbafa43ba562202724c"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.646170 26360 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:25.647018 26375 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.647207 26360 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:25.647323 26360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "e4f4bb6c4de745f0b184ca7383dc0b08"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "f30f4f8c8a51cba24c026c7eb9e779a3"
server_key_iv: "0e1c56a4c8f98dbafa43ba562202724c"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.647404 26360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:25.668000 26360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:25.670203 26360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:25.670388 26360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:25.670964 26360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:25.671029 26360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.671079 26360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:25.671103 26360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:25.678596 26360 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:38049
I20260811 14:30:25.678902 26511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:38049 every 8 connection(s)
I20260811 14:30:25.679047 26360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:25.682097 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.671542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:51843 (local address 127.23.186.254:41799)
0811 14:30:25.671627 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:25.671631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:25.671805 (+ 174us) server_negotiation.cc:407] Connection header received
0811 14:30:25.672306 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:25.672308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:25.672322 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:25.672404 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:25.673279 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.673794 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.675464 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.675870 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.675962 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.679051 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:25.679052 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:25.679053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:25.679062 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:25.680854 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.681277 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.681278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.681278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.681335 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.681567 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.681568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.681568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.681684 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:25.681735 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:25.681955 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 14:30:25.682031 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:30:25.682281 26386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.671460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.23.186.193:51843)
0811 14:30:25.671666 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:25.671708 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:25.672200 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:25.672545 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:25.672549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:25.672909 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:25.673173 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.673178 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.673896 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.673897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.675347 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.675351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.675985 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.675986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.676149 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.676792 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:25.676801 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:25.678924 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:25.680941 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.680949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.680951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.681197 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.681391 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.681391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.681392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.681511 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.681724 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:25.681727 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:25.681850 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 14:30:25.681930 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:30:25.682066 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260811 14:30:25.682904 26512 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41799
I20260811 14:30:25.683005 26512 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:25.683207 26512 heartbeater.cc:511] Master 127.23.186.254:41799 requested a full tablet report, sending...
I20260811 14:30:25.683760 26276 ts_manager.cc:194] Registered new tserver with Master: e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049)
I20260811 14:30:25.684784 26276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:51843
I20260811 14:30:25.688920 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 26360
I20260811 14:30:25.689035 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:25.689254 24299 external_mini_cluster.cc:1428] Setting key d92565a6a07be1886628465493cd5389
I20260811 14:30:25.690304 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.684271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:36805 (local address 127.23.186.254:41799)
0811 14:30:25.684313 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:25.684315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:25.684385 (+ 70us) server_negotiation.cc:407] Connection header received
0811 14:30:25.684462 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:25.684462 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:25.684470 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:25.684510 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:25.684953 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.685672 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.686352 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.686555 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.686607 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.687505 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:25.687506 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:25.687507 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:25.687513 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:25.689113 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.689509 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.689510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.689511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.689591 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.689806 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.689807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.689807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.689903 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:25.689949 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:25.690121 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 14:30:25.690208 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:30:25.690328 26518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.684178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.23.186.193:36805)
0811 14:30:25.684323 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:25.684334 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:25.684395 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:25.684534 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:25.684534 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:25.684772 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:25.684844 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.684846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.685770 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.685771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.686280 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.686282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.686623 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.686624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.686687 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.687107 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:25.687112 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:25.687404 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:25.689194 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.689194 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.689195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.689408 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.689666 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.689667 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.689668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.689744 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.689961 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:25.689962 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:25.690010 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 14:30:25.690080 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:30:25.690215 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:25.721029 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41799
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:25.807197 26521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:25.807457 26521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:25.807504 26521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:25.808753 26521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:25.808828 26521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:25.808864 26521 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 14:30:25.808890 26521 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:25.808930 26521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:25.810460 26521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:25.810761 26521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:25.811017 26521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:25.816385 26529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.816434 26527 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.816419 26526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:25.816681 26521 server_base.cc:1034] running on GCE node
I20260811 14:30:25.816902 26521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:25.817116 26521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:25.818315 26521 hybrid_clock.cc:648] HybridClock initialized: now 1786458625818275 us; error 51 us; skew 500 ppm
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:25.821153 26521 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:25.821655 26521 webserver.cc:460] Webserver started at http://127.23.186.194:36337/ using document root <none> and password file <none>
I20260811 14:30:25.821898 26521 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:25.821969 26521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:25.822132 26521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:25.823257 26521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "07e17b40b92b4e10a70876f5166c7dd1"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "22a456cd8bb73894ee8411e1d54a5c3c"
server_key_iv: "f603ff0813a9db4e20689937f689b186"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.823652 26521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "07e17b40b92b4e10a70876f5166c7dd1"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "22a456cd8bb73894ee8411e1d54a5c3c"
server_key_iv: "f603ff0813a9db4e20689937f689b186"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.825166 26521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:25.826015 26537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.826212 26521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:25.826325 26521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "07e17b40b92b4e10a70876f5166c7dd1"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "22a456cd8bb73894ee8411e1d54a5c3c"
server_key_iv: "f603ff0813a9db4e20689937f689b186"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.826403 26521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:25.836632 26521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:25.839143 26521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:25.839323 26521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:25.839879 26521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:25.839977 26521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.840024 26521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:25.840080 26521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.847591 26521 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:44041
I20260811 14:30:25.847707 26668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:44041 every 8 connection(s)
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:25.848057 26521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:25.851343 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.840612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:48063 (local address 127.23.186.254:41799)
0811 14:30:25.840672 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:25.840674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:25.840918 (+ 244us) server_negotiation.cc:407] Connection header received
0811 14:30:25.841536 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:25.841539 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:25.841554 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:25.841630 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:25.842562 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.843437 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.845254 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.845550 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.845612 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.848754 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:25.848755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:25.848756 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:25.848765 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:25.850008 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.850493 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.850495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.850495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.850563 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.850793 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.850794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.850794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.850911 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:25.850963 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:25.851209 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 14:30:25.851277 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:30:25.851565 26548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.840592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.23.186.194:48063)
0811 14:30:25.840795 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:25.840831 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:25.841377 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:25.841739 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:25.841743 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:25.842136 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:25.842444 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.842450 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.843568 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.843569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.845112 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.845115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.845649 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.845651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.845826 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.846518 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:25.846531 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:25.848596 (+ 2065us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:25.850088 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.850095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.850097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.850338 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.850626 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.850626 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.850627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.850735 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.850968 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:25.850973 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:25.851113 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 14:30:25.851199 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:30:25.851334 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":123,"threads_started":1}
I20260811 14:30:25.852144 26669 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41799
I20260811 14:30:25.852279 26669 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:25.852480 26669 heartbeater.cc:511] Master 127.23.186.254:41799 requested a full tablet report, sending...
I20260811 14:30:25.852975 26277 ts_manager.cc:194] Registered new tserver with Master: 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041)
I20260811 14:30:25.853518 26277 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:48063
I20260811 14:30:25.857318 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 26521
I20260811 14:30:25.857441 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:25.857630 24299 external_mini_cluster.cc:1428] Setting key 088e7ce7a19d12bec4ae3bcbff607616
I20260811 14:30:25.859177 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.853356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:39959 (local address 127.23.186.254:41799)
0811 14:30:25.853397 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:25.853398 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:25.853532 (+ 134us) server_negotiation.cc:407] Connection header received
0811 14:30:25.853608 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:25.853609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:25.853620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:25.853668 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:25.854052 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.854536 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.855232 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.855447 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.855523 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.856404 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:25.856405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:25.856405 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:25.856413 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:25.858005 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.858373 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.858374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.858375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.858443 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:25.858665 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:25.858665 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:25.858665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:25.858816 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:25.858871 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:25.859049 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 14:30:25.859115 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:30:25.859360 26676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:25.853287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.23.186.194:39959)
0811 14:30:25.853465 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:25.853476 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:25.853549 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:25.853728 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:25.853730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:25.853909 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:25.853960 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.853961 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.854629 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.854630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.855144 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:25.855145 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:25.855521 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:25.855522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:25.855609 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:25.856019 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:25.856023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:25.856310 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:25.858087 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.858088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.858089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.858294 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.858517 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:25.858518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:25.858518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:25.858596 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:25.858886 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:25.858887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:25.858941 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:30:25.859016 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:30:25.859143 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":80,"threads_started":1}
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:25.887790 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41799
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:25.976132 26679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:25.976428 26679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:25.976488 26679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:25.977850 26679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:25.977934 26679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:25.977972 26679 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 14:30:25.977991 26679 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:25.978030 26679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:25.979903 26679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:25.980212 26679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:25.980470 26679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:25.986047 26685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.986012 26686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:25.986007 26688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:25.986305 26679 server_base.cc:1034] running on GCE node
I20260811 14:30:25.986552 26679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:25.986769 26679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:25.987917 26679 hybrid_clock.cc:648] HybridClock initialized: now 1786458625987906 us; error 28 us; skew 500 ppm
Aug 11 14:30:25 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:25.990641 26679 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:25.991065 26679 webserver.cc:460] Webserver started at http://127.23.186.195:44919/ using document root <none> and password file <none>
I20260811 14:30:25.991359 26679 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:25.991431 26679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:25.991608 26679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:25.992871 26679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "791d09c678fc4e2db789c1dab27c91ae"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "482353a3e7afd10955f8ea2556c97231"
server_key_iv: "4ae304d53dccdcf3c582eda66dd5ebdd"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.993201 26679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "791d09c678fc4e2db789c1dab27c91ae"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "482353a3e7afd10955f8ea2556c97231"
server_key_iv: "4ae304d53dccdcf3c582eda66dd5ebdd"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.994563 26679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:25.995273 26695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:25.995460 26679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:25.995535 26679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "791d09c678fc4e2db789c1dab27c91ae"
format_stamp: "Formatted at 2026-08-11 14:30:25 on dist-test-slave-cbsf"
server_key: "482353a3e7afd10955f8ea2556c97231"
server_key_iv: "4ae304d53dccdcf3c582eda66dd5ebdd"
server_key_version: "encryptionkey@0"
I20260811 14:30:25.995604 26679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:26.009761 26679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:26.012229 26679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:26.012355 26679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:26.013115 26679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:26.013180 26679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.013206 26679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:26.013219 26679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.020753 26679 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:38155
Aug 11 14:30:26 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:26.020821 26827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:38155 every 8 connection(s)
I20260811 14:30:26.021175 26679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:26.024113 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 26679
I20260811 14:30:26.024226 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:26.024250 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.013611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:40531 (local address 127.23.186.254:41799)
0811 14:30:26.013692 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.013694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.013921 (+ 227us) server_negotiation.cc:407] Connection header received
0811 14:30:26.014477 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.014479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.014495 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.014574 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:26.015503 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.016050 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.017852 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.018236 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.018319 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.021398 (+ 3079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:26.021399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:26.021399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:26.021410 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:26.022753 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.023186 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.023187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.023188 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.023257 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.023517 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.023518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.023519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.023658 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:26.023733 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.024035 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.024157 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:30:26.024415 26706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.013580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.23.186.195:40531)
0811 14:30:26.013788 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.013834 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.014366 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.014664 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.014668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.015041 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:26.015400 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.015407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.016165 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.016166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.017719 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.017724 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.018367 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.018368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.018541 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.019234 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:26.019266 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:26.021272 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:26.022861 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:26.022868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:26.022870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:26.023103 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:26.023335 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:26.023336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:26.023336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:26.023450 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:26.023735 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:26.023742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:26.023892 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.023998 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.024177 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":130,"threads_started":1}
I20260811 14:30:26.024430 24299 external_mini_cluster.cc:1428] Setting key 62097989cd85fb237fd2c00f7ce3581b
I20260811 14:30:26.025102 26828 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41799
I20260811 14:30:26.025199 26828 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:26.025406 26828 heartbeater.cc:511] Master 127.23.186.254:41799 requested a full tablet report, sending...
I20260811 14:30:26.025863 26276 ts_manager.cc:194] Registered new tserver with Master: 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:26.026422 26276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:40531
I20260811 14:30:26.032282 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.026252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.23.186.195:49909)
0811 14:30:26.026622 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.026636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.026700 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.026862 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.026863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.027050 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:26.027108 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.027110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.027782 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.027783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.028308 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.028309 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.028734 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.028735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.028803 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.029209 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:26.029214 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:26.029608 (+ 394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:26.031167 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:26.031168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:26.031169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:26.031345 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:26.031585 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:26.031586 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:26.031586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:26.031653 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:26.031911 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:26.031912 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:26.031962 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.032065 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.032180 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":67,"threads_started":1}
I20260811 14:30:26.032277 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.026324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:49909 (local address 127.23.186.254:41799)
0811 14:30:26.026363 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.026365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.026689 (+ 324us) server_negotiation.cc:407] Connection header received
0811 14:30:26.026761 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.026762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.026770 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.026837 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:26.027198 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.027685 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.028400 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.028622 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.028698 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.029699 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:26.029701 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:26.029701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:26.029709 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:26.031065 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.031434 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.031435 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.031436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.031495 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.031720 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.031722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.031722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.031844 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:26.031921 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.032095 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.032178 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260811 14:30:26.036293 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:30:26 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:30:26 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458626, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:26.062652 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.055145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34680 (local address 127.23.186.254:41799)
0811 14:30:26.055216 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.055218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.055227 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:26.055259 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.055259 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.055269 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.055348 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:26.055969 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.056490 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.057346 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.057562 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.057642 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.060046 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:26.060048 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:26.060049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:26.060057 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:26.061349 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.061774 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.061775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.061776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.061841 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.062055 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.062056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.062057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.062186 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:26.062298 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.062441 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.062533 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:30:26.064424 26276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:26.066166 26276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:26.073482 26850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.069188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:44041 (local address 127.0.0.1:49408)
0811 14:30:26.069583 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.069635 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.069712 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.069990 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.069994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.069997 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.070108 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.070110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.071114 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.071116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.072259 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.072261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.073146 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.073147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.073242 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.073258 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.073337 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.073395 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":71,"threads_started":1}
I20260811 14:30:26.073521 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.069318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.23.186.194:44041)
0811 14:30:26.069683 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.069687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.069700 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:30:26.069804 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.069806 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.069823 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.069901 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.070199 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.071007 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.072348 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.073054 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.073135 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.073194 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.073365 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.073408 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":85,"threads_started":1}
I20260811 14:30:26.074103 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.069192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38049 (local address 127.0.0.1:47690)
0811 14:30:26.069813 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.069824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.069922 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.070286 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.070287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.070289 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.070334 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.070336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.071643 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.071644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.072856 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.072857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.073862 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.073863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.073926 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.073930 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.073991 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.074034 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":108,"threads_started":1}
I20260811 14:30:26.074261 26849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.069418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.0.0.1:46818)
0811 14:30:26.069824 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.069832 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.069879 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.070152 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.070153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.070155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.070191 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.070192 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.071512 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.071513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.072743 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.072746 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.074003 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.074004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.074070 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.074076 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.074132 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.074181 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":43,"thread_start_us":71,"threads_started":1}
I20260811 14:30:26.074558 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.069549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.23.186.195:38155)
0811 14:30:26.069720 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.069725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.069935 (+ 210us) server_negotiation.cc:407] Connection header received
0811 14:30:26.069981 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.069983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.069999 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.070069 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.070258 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.071419 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.072843 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.073615 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.073696 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.073757 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.074387 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.074450 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":101,"threads_started":1}
I20260811 14:30:26.074487 26595 tablet_service.cc:1467] Processing CreateTablet for tablet a997fb66691f4bc7ac475e255af8ef30 (DEFAULT_TABLE table=test-table [id=7c608f8c70ab444faa9be9f4ae208a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:26.074710 26852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.069348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47690 (local address 127.23.186.193:38049)
0811 14:30:26.069584 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.069590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.069889 (+ 299us) server_negotiation.cc:407] Connection header received
0811 14:30:26.070058 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.070064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.070083 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.070142 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.070428 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.071551 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.072948 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.073711 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.073775 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.073828 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.074481 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.074531 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":108,"threads_started":1}
I20260811 14:30:26.074910 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a997fb66691f4bc7ac475e255af8ef30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.075261 26754 tablet_service.cc:1467] Processing CreateTablet for tablet a997fb66691f4bc7ac475e255af8ef30 (DEFAULT_TABLE table=test-table [id=7c608f8c70ab444faa9be9f4ae208a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:26.075351 26434 tablet_service.cc:1467] Processing CreateTablet for tablet a997fb66691f4bc7ac475e255af8ef30 (DEFAULT_TABLE table=test-table [id=7c608f8c70ab444faa9be9f4ae208a80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:26.075634 26754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a997fb66691f4bc7ac475e255af8ef30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.075670 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a997fb66691f4bc7ac475e255af8ef30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.078147 26855 tablet_bootstrap.cc:492] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Bootstrap starting.
I20260811 14:30:26.078956 26855 tablet_bootstrap.cc:654] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.079303 26855 log.cc:824] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:26.079567 26856 tablet_bootstrap.cc:492] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Bootstrap starting.
I20260811 14:30:26.080413 26855 tablet_bootstrap.cc:492] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: No bootstrap required, opened a new log
I20260811 14:30:26.080494 26855 ts_tablet_manager.cc:1397] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:26.080755 26856 tablet_bootstrap.cc:654] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.080809 26858 tablet_bootstrap.cc:492] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Bootstrap starting.
I20260811 14:30:26.081100 26856 log.cc:824] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:26.081521 26858 tablet_bootstrap.cc:654] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.081841 26858 log.cc:824] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:26.082088 26856 tablet_bootstrap.cc:492] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: No bootstrap required, opened a new log
I20260811 14:30:26.082162 26856 ts_tablet_manager.cc:1397] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:30:26.082746 26855 raft_consensus.cc:348] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.082938 26855 raft_consensus.cc:374] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.082975 26855 raft_consensus.cc:729] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e17b40b92b4e10a70876f5166c7dd1, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.083103 26855 consensus_queue.cc:260] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.083407 26855 ts_tablet_manager.cc:1428] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:26.083461 26669 heartbeater.cc:503] Master 127.23.186.254:41799 was elected leader, sending a full tablet report...
I20260811 14:30:26.084105 26858 tablet_bootstrap.cc:492] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: No bootstrap required, opened a new log
I20260811 14:30:26.084179 26858 ts_tablet_manager.cc:1397] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 14:30:26.084561 26856 raft_consensus.cc:348] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.084755 26856 raft_consensus.cc:374] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.084810 26856 raft_consensus.cc:729] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.084940 26856 consensus_queue.cc:260] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.085246 26856 ts_tablet_manager.cc:1428] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 14:30:26.085290 26828 heartbeater.cc:503] Master 127.23.186.254:41799 was elected leader, sending a full tablet report...
I20260811 14:30:26.086374 26858 raft_consensus.cc:348] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.086550 26858 raft_consensus.cc:374] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.086598 26858 raft_consensus.cc:729] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.086730 26858 consensus_queue.cc:260] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.087011 26858 ts_tablet_manager.cc:1428] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:26.087044 26512 heartbeater.cc:503] Master 127.23.186.254:41799 was elected leader, sending a full tablet report...
W20260811 14:30:26.098603 26670 tablet.cc:2369] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:30:26.180300 26513 tablet.cc:2369] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:30:26.271831 26829 tablet.cc:2369] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:26.279613 26865 raft_consensus.cc:482] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:26.279770 26865 raft_consensus.cc:504] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.280272 26865 leader_election.cc:288] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041), 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:26.284507 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.280744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:33757 (local address 127.23.186.195:38155)
0811 14:30:26.280790 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.280792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.280803 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:26.280845 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.280846 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.280855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.280907 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.281086 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.281870 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.283506 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.284144 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.284207 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.284215 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.284379 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.284442 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:26.284507 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.280584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:49595 (local address 127.23.186.194:44041)
0811 14:30:26.280672 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.280674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.280709 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:30:26.280778 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.280779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.280790 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.280860 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.281094 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.281868 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.283506 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.284104 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.284191 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.284203 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.284380 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.284429 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 14:30:26.284556 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.280511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.23.186.193:33757)
0811 14:30:26.280730 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.280740 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.280782 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.280923 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.280924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.280925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.281008 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.281009 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.281971 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.281972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.283385 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.283389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.284218 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.284218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.284286 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.284293 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.284357 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.284424 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":108,"thread_start_us":96,"threads_started":1}
I20260811 14:30:26.284556 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.280471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:44041 (local address 127.23.186.193:49595)
0811 14:30:26.280600 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.280616 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.280684 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.280874 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.280875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.280877 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.280983 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.280986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.281968 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.281969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.283385 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.283389 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.284200 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.284202 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.284286 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.284293 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.284357 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.284424 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":79,"thread_start_us":77,"threads_started":1}
I20260811 14:30:26.285048 26618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a997fb66691f4bc7ac475e255af8ef30" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1" is_pre_election: true
I20260811 14:30:26.285179 26778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a997fb66691f4bc7ac475e255af8ef30" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791d09c678fc4e2db789c1dab27c91ae" is_pre_election: true
I20260811 14:30:26.285240 26618 raft_consensus.cc:2454] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 0.
I20260811 14:30:26.285367 26778 raft_consensus.cc:2454] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 0.
I20260811 14:30:26.285524 26377 leader_election.cc:302] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, e4f4bb6c4de745f0b184ca7383dc0b08; no voters:
I20260811 14:30:26.285683 26865 raft_consensus.cc:2781] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:26.285755 26865 raft_consensus.cc:482] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:26.285773 26865 raft_consensus.cc:3037] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.286506 26865 raft_consensus.cc:504] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.286671 26865 leader_election.cc:288] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 election: Requested vote from peers 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041), 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:26.286826 26618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a997fb66691f4bc7ac475e255af8ef30" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1"
I20260811 14:30:26.286846 26778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a997fb66691f4bc7ac475e255af8ef30" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791d09c678fc4e2db789c1dab27c91ae"
I20260811 14:30:26.286934 26618 raft_consensus.cc:3037] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.286929 26778 raft_consensus.cc:3037] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.287642 26778 raft_consensus.cc:2454] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 1.
I20260811 14:30:26.287758 26618 raft_consensus.cc:2454] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 1.
I20260811 14:30:26.287833 26376 leader_election.cc:302] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 791d09c678fc4e2db789c1dab27c91ae, e4f4bb6c4de745f0b184ca7383dc0b08; no voters:
I20260811 14:30:26.287938 26865 raft_consensus.cc:2781] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:26.288075 26865 raft_consensus.cc:686] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 LEADER]: Becoming Leader. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Running, Role: LEADER
I20260811 14:30:26.288220 26865 consensus_queue.cc:237] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.289134 26276 catalog_manager.cc:5495] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193). New cstate: current_term: 1 leader_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } health_report { overall_health: UNKNOWN } } }
Aug 11 14:30:26 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458625, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:26.324807 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.317521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41799 (local address 127.0.0.1:34690)
0811 14:30:26.317771 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.317790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.317881 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.318043 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.318044 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.318398 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:26.318449 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.318452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.319142 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.319143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.319671 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.319673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.320056 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.320057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.320111 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.320658 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:26.320663 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:26.322146 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:26.323774 (+ 1628us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:26.323782 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:26.323784 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:26.323977 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:26.324161 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:26.324162 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:26.324162 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:26.324219 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:26.324435 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:26.324442 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:26.324487 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.324542 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.324642 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":143,"threads_started":1}
I20260811 14:30:26.324857 26334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.317628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34690 (local address 127.23.186.254:41799)
0811 14:30:26.317728 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.317731 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.317860 (+ 129us) server_negotiation.cc:407] Connection header received
0811 14:30:26.317940 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.317941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.317954 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.318029 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:26.318559 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.319053 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.319762 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.319945 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.320057 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.322247 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:26.322248 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:26.322248 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:26.322262 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:26.323696 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.324047 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.324048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.324048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.324102 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:26.324271 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:26.324272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:26.324272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:26.324382 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:26.324433 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.324591 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.324744 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":67}
I20260811 14:30:26.326999 26276 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:34690:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 14:30:26.327220 26276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:26.328788 26595 tablet_service.cc:1467] Processing CreateTablet for tablet f8a24ac7641a48f19a657a8817e93aed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0802e39db8574336a87f657c90d824ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:26.328820 26434 tablet_service.cc:1467] Processing CreateTablet for tablet f8a24ac7641a48f19a657a8817e93aed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0802e39db8574336a87f657c90d824ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:26.328984 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8a24ac7641a48f19a657a8817e93aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.328990 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8a24ac7641a48f19a657a8817e93aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.330447 26858 tablet_bootstrap.cc:492] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: Bootstrap starting.
I20260811 14:30:26.330449 26855 tablet_bootstrap.cc:492] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: Bootstrap starting.
I20260811 14:30:26.330590 26754 tablet_service.cc:1467] Processing CreateTablet for tablet f8a24ac7641a48f19a657a8817e93aed (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0802e39db8574336a87f657c90d824ba]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:26.330772 26754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8a24ac7641a48f19a657a8817e93aed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.330909 26858 tablet_bootstrap.cc:654] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.331198 26855 tablet_bootstrap.cc:654] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.331905 26858 tablet_bootstrap.cc:492] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: No bootstrap required, opened a new log
I20260811 14:30:26.331939 26855 tablet_bootstrap.cc:492] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: No bootstrap required, opened a new log
I20260811 14:30:26.331960 26858 ts_tablet_manager.cc:1397] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:26.331979 26855 ts_tablet_manager.cc:1397] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:26.332124 26858 raft_consensus.cc:348] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.332135 26855 raft_consensus.cc:348] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.332201 26858 raft_consensus.cc:374] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.332211 26855 raft_consensus.cc:374] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.332227 26858 raft_consensus.cc:729] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.332232 26855 raft_consensus.cc:729] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e17b40b92b4e10a70876f5166c7dd1, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.332285 26855 consensus_queue.cc:260] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.332285 26858 consensus_queue.cc:260] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.332374 26858 ts_tablet_manager.cc:1428] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.332374 26855 ts_tablet_manager.cc:1428] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.332377 26856 tablet_bootstrap.cc:492] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: Bootstrap starting.
I20260811 14:30:26.332459 26862 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.332584 26862 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.332899 26865 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.333012 26865 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.333518 26856 tablet_bootstrap.cc:654] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.334236 26856 tablet_bootstrap.cc:492] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: No bootstrap required, opened a new log
I20260811 14:30:26.334280 26856 ts_tablet_manager.cc:1397] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:30:26.334439 26856 raft_consensus.cc:348] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.334497 26856 raft_consensus.cc:374] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.334511 26856 raft_consensus.cc:729] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.334580 26856 consensus_queue.cc:260] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.334692 26856 ts_tablet_manager.cc:1428] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.334753 26864 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.334859 26864 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.610500 26864 raft_consensus.cc:482] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:26.610625 26864 raft_consensus.cc:504] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.611088 26864 leader_election.cc:288] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049), 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041)
I20260811 14:30:26.615383 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.611405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:59915 (local address 127.23.186.193:38049)
0811 14:30:26.611524 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.611533 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.611550 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:30:26.611646 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.611647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.611658 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.611738 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.611942 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.612769 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.614380 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.614977 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.615054 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.615065 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.615253 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.615302 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":51,"threads_started":1}
I20260811 14:30:26.615382 26851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.611524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53777 (local address 127.23.186.194:44041)
0811 14:30:26.611573 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.611575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.611584 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:26.611624 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.611625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.611636 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.611716 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:26.611941 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.612760 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.614380 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.614962 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.615056 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.615065 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.615258 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.615306 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:30:26.615418 26882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.611322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38049 (local address 127.23.186.195:59915)
0811 14:30:26.611472 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.611487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.611560 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.611748 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.611749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.611753 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.611862 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.611863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.612873 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.612875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.614269 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.614273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.615071 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.615072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.615162 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.615168 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.615230 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.615304 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":97,"thread_start_us":102,"threads_started":1}
I20260811 14:30:26.615418 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.611331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:44041 (local address 127.23.186.195:53777)
0811 14:30:26.611497 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.611517 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.611560 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.611747 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.611749 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.611751 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:26.611849 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.611851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.612873 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.612875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.614269 (+ 1394us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.614273 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.615070 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.615072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.615162 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.615170 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.615230 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.615305 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":109,"thread_start_us":99,"threads_started":1}
I20260811 14:30:26.615875 26460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a24ac7641a48f19a657a8817e93aed" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" is_pre_election: true
I20260811 14:30:26.616034 26460 raft_consensus.cc:2454] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 0.
I20260811 14:30:26.616099 26618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a24ac7641a48f19a657a8817e93aed" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1" is_pre_election: true
I20260811 14:30:26.616227 26618 raft_consensus.cc:2454] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 0.
I20260811 14:30:26.616287 26696 leader_election.cc:302] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 791d09c678fc4e2db789c1dab27c91ae, e4f4bb6c4de745f0b184ca7383dc0b08; no voters:
I20260811 14:30:26.616439 26864 raft_consensus.cc:2781] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:26.616493 26864 raft_consensus.cc:482] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:26.616526 26864 raft_consensus.cc:3037] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.617300 26864 raft_consensus.cc:504] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.617470 26864 leader_election.cc:288] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 election: Requested vote from peers e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049), 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041)
I20260811 14:30:26.617638 26618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a24ac7641a48f19a657a8817e93aed" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1"
I20260811 14:30:26.617725 26618 raft_consensus.cc:3037] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.618357 26618 raft_consensus.cc:2454] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 1.
I20260811 14:30:26.618500 26460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8a24ac7641a48f19a657a8817e93aed" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08"
I20260811 14:30:26.618613 26460 raft_consensus.cc:3037] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.618587 26697 leader_election.cc:302] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, 791d09c678fc4e2db789c1dab27c91ae; no voters:
I20260811 14:30:26.618703 26864 raft_consensus.cc:2781] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:26.618844 26864 raft_consensus.cc:686] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [term 1 LEADER]: Becoming Leader. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Running, Role: LEADER
I20260811 14:30:26.618970 26864 consensus_queue.cc:237] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.619468 26885 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: TxnStatusTablet state changed. Reason: New leader 791d09c678fc4e2db789c1dab27c91ae. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.619570 26885 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:26.619606 26460 raft_consensus.cc:2454] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 1.
I20260811 14:30:26.619872 26887 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 14:30:26.619865 26276 catalog_manager.cc:5495] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195). New cstate: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260811 14:30:26.620024 26887 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 14:30:26.688027 26877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.685317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.0.0.1:46820)
0811 14:30:26.685401 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.685414 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.685490 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.685695 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.685696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.685698 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:26.685754 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.685757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.686446 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.686447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.687046 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.687057 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.687449 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.687450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.687524 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.687546 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.687829 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.687842 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.687900 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.687955 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:30:26.688050 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.685414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46820 (local address 127.23.186.195:38155)
0811 14:30:26.685470 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.685473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.685483 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:26.685587 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.685588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.685607 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.685686 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:26.685828 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.686356 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.687153 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.687371 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.687449 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.687645 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.687760 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.687853 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.687907 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.687937 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:30:26.689985 26460 raft_consensus.cc:1261] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Refusing update from remote peer 791d09c678fc4e2db789c1dab27c91ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:26.690009 26618 raft_consensus.cc:1261] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Refusing update from remote peer 791d09c678fc4e2db789c1dab27c91ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:26.690393 26864 consensus_queue.cc:1035] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:26.690562 26885 consensus_queue.cc:1035] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:26.691833 26865 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: TxnStatusTablet state changed. Reason: New leader 791d09c678fc4e2db789c1dab27c91ae. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.691941 26865 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.693136 26862 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: TxnStatusTablet state changed. Reason: New leader 791d09c678fc4e2db789c1dab27c91ae. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.693246 26862 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.693980 26864 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.694116 26864 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:26.694195 26890 mvcc.cc:205] Tried to move back new op lower bound from 7317334534919401472 to 7317334534631952384. Current Snapshot: MvccSnapshot[applied={T|T < 7317334534919401472}]
I20260811 14:30:26.694538 26865 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.694633 26865 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P e4f4bb6c4de745f0b184ca7383dc0b08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.695525 26864 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.695611 26864 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 791d09c678fc4e2db789c1dab27c91ae: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:26.696130 26862 tablet_replica.cc:439] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } } }
I20260811 14:30:26.696233 26862 tablet_replica.cc:442] T f8a24ac7641a48f19a657a8817e93aed P 07e17b40b92b4e10a70876f5166c7dd1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:26.696477 26894 mvcc.cc:205] Tried to move back new op lower bound from 7317334534919401472 to 7317334534631952384. Current Snapshot: MvccSnapshot[applied={T|T < 7317334534919401472}]
I20260811 14:30:26.699389 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.696017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.23.186.193:38049)
0811 14:30:26.696095 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.696097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.696868 (+ 771us) server_negotiation.cc:407] Connection header received
0811 14:30:26.696969 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.696970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.696990 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.697059 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:26.697232 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.697728 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.698543 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.698779 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.698869 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.699041 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.699152 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.699238 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.699275 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.699302 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 14:30:26.704980 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.702511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:48047 (local address 127.23.186.195:38155)
0811 14:30:26.702582 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.702584 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.702594 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:26.702676 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.702677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.702691 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.702756 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:26.702901 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.703376 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.704194 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.704395 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.704472 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.704682 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.704752 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.704814 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.704872 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.704897 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:30:26.705005 26904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.702387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.23.186.193:48047)
0811 14:30:26.702542 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.702555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.702610 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.702758 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.702759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.702760 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:26.702830 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.702831 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.703460 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.703461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.704098 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.704100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.704484 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.704485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.704555 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.704578 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.704817 (+ 239us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.704820 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.704869 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.704917 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":86,"threads_started":1}
I20260811 14:30:26.707756 26618 raft_consensus.cc:1261] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Refusing update from remote peer e4f4bb6c4de745f0b184ca7383dc0b08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:26.707860 26778 raft_consensus.cc:1261] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Refusing update from remote peer e4f4bb6c4de745f0b184ca7383dc0b08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:26.708088 26870 consensus_queue.cc:1035] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:26.708236 26865 consensus_queue.cc:1035] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:26.717491 26883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.714781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:35279 (local address 127.23.186.193:38049)
0811 14:30:26.714837 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.714840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.714888 (+ 48us) server_negotiation.cc:407] Connection header received
0811 14:30:26.714959 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.714960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.714969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.715009 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:26.715159 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.715625 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.716835 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.717022 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.717081 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.717287 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.717313 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.717347 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.717415 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.717433 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:30:26.717787 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.714583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38049 (local address 127.23.186.195:35279)
0811 14:30:26.714824 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.714837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.714900 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.715026 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.715027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.715028 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:26.715099 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.715101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.715710 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.715711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.716762 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.716765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.717101 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.717102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.717188 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.717205 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.717362 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.717364 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.717414 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.717462 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":96,"threads_started":1}
I20260811 14:30:26.727020 26275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:26.727161 26275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:26.728065 26275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Sending DeleteTablet for 3 replicas of tablet a997fb66691f4bc7ac475e255af8ef30
I20260811 14:30:26.728646 26595 tablet_service.cc:1514] Processing DeleteTablet for tablet a997fb66691f4bc7ac475e255af8ef30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:26 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:49408
I20260811 14:30:26.728945 26754 tablet_service.cc:1514] Processing DeleteTablet for tablet a997fb66691f4bc7ac475e255af8ef30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:26 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260811 14:30:26.728983 26916 tablet_replica.cc:330] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: stopping tablet replica
I20260811 14:30:26.729228 26916 raft_consensus.cc:2229] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:26.729259 26434 tablet_service.cc:1514] Processing DeleteTablet for tablet a997fb66691f4bc7ac475e255af8ef30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:26 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:47690
I20260811 14:30:26.729413 26916 raft_consensus.cc:2258] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:26.729811 26275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:26.729980 26275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:26.730186 26916 ts_tablet_manager.cc:1905] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:26.731477 26754 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba4f73be3b4c4cad77f5f702c9bdae (DEFAULT_TABLE table=test-table [id=c243e67b484f476e9e8b7819c36106f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:26.731647 26754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba4f73be3b4c4cad77f5f702c9bdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.732285 26916 ts_tablet_manager.cc:1918] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:26.732445 26916 log.cc:1196] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/a997fb66691f4bc7ac475e255af8ef30
I20260811 14:30:26.732839 26916 ts_tablet_manager.cc:1939] T a997fb66691f4bc7ac475e255af8ef30 P 07e17b40b92b4e10a70876f5166c7dd1: Deleting consensus metadata
I20260811 14:30:26.733166 26856 tablet_bootstrap.cc:492] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Bootstrap starting.
I20260811 14:30:26.733613 26856 tablet_bootstrap.cc:654] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.734192 26261 catalog_manager.cc:4841] TS 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041): tablet a997fb66691f4bc7ac475e255af8ef30 (table test-table [id=7c608f8c70ab444faa9be9f4ae208a80]) successfully deleted
I20260811 14:30:26.734380 26856 tablet_bootstrap.cc:492] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: No bootstrap required, opened a new log
I20260811 14:30:26.734429 26856 ts_tablet_manager.cc:1397] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:26.734413 26595 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba4f73be3b4c4cad77f5f702c9bdae (DEFAULT_TABLE table=test-table [id=c243e67b484f476e9e8b7819c36106f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:26.734531 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba4f73be3b4c4cad77f5f702c9bdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.734598 26856 raft_consensus.cc:348] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.734675 26856 raft_consensus.cc:374] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.734695 26856 raft_consensus.cc:729] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.734740 26856 consensus_queue.cc:260] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.734831 26856 ts_tablet_manager.cc:1428] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.735090 26918 tablet_replica.cc:330] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: stopping tablet replica
I20260811 14:30:26.735235 26918 raft_consensus.cc:2229] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:26.735380 26918 raft_consensus.cc:2258] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:26.735790 26855 tablet_bootstrap.cc:492] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Bootstrap starting.
I20260811 14:30:26.736228 26855 tablet_bootstrap.cc:654] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.737252 26434 tablet_service.cc:1467] Processing CreateTablet for tablet a1ba4f73be3b4c4cad77f5f702c9bdae (DEFAULT_TABLE table=test-table [id=c243e67b484f476e9e8b7819c36106f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:26.737385 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a1ba4f73be3b4c4cad77f5f702c9bdae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:26.738837 26858 tablet_bootstrap.cc:492] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Bootstrap starting.
I20260811 14:30:26.739280 26858 tablet_bootstrap.cc:654] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:26.739975 26858 tablet_bootstrap.cc:492] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: No bootstrap required, opened a new log
I20260811 14:30:26.740024 26858 ts_tablet_manager.cc:1397] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:26.740065 26855 tablet_bootstrap.cc:492] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: No bootstrap required, opened a new log
I20260811 14:30:26.740116 26855 ts_tablet_manager.cc:1397] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 14:30:26.740175 26858 raft_consensus.cc:348] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.740252 26858 raft_consensus.cc:374] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.740271 26858 raft_consensus.cc:729] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.740260 26855 raft_consensus.cc:348] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.740330 26855 raft_consensus.cc:374] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:26.740314 26858 consensus_queue.cc:260] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.740350 26855 raft_consensus.cc:729] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e17b40b92b4e10a70876f5166c7dd1, State: Initialized, Role: FOLLOWER
I20260811 14:30:26.740397 26858 ts_tablet_manager.cc:1428] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:26.740396 26855 consensus_queue.cc:260] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.740483 26855 ts_tablet_manager.cc:1428] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:26.741894 26918 ts_tablet_manager.cc:1905] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:26.743729 26918 ts_tablet_manager.cc:1918] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:26.743815 26918 log.cc:1196] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/a997fb66691f4bc7ac475e255af8ef30
I20260811 14:30:26.744138 26918 ts_tablet_manager.cc:1939] T a997fb66691f4bc7ac475e255af8ef30 P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting consensus metadata
I20260811 14:30:26.744598 26917 tablet_replica.cc:330] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: stopping tablet replica
I20260811 14:30:26.744668 26260 catalog_manager.cc:4841] TS e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049): tablet a997fb66691f4bc7ac475e255af8ef30 (table test-table [id=7c608f8c70ab444faa9be9f4ae208a80]) successfully deleted
I20260811 14:30:26.744755 26917 raft_consensus.cc:2229] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:26.744822 26917 raft_consensus.cc:2258] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:26.745314 26917 ts_tablet_manager.cc:1905] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:26.747129 26917 ts_tablet_manager.cc:1918] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:26.747205 26917 log.cc:1196] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/a997fb66691f4bc7ac475e255af8ef30
I20260811 14:30:26.747479 26917 ts_tablet_manager.cc:1939] T a997fb66691f4bc7ac475e255af8ef30 P 791d09c678fc4e2db789c1dab27c91ae: Deleting consensus metadata
I20260811 14:30:26.747911 26260 catalog_manager.cc:4841] TS 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155): tablet a997fb66691f4bc7ac475e255af8ef30 (table test-table [id=7c608f8c70ab444faa9be9f4ae208a80]) successfully deleted
I20260811 14:30:26.762161 26864 raft_consensus.cc:482] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:26.762277 26864 raft_consensus.cc:504] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.762436 26864 leader_election.cc:288] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049), 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041)
I20260811 14:30:26.762691 26617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba4f73be3b4c4cad77f5f702c9bdae" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1" is_pre_election: true
I20260811 14:30:26.762712 26460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba4f73be3b4c4cad77f5f702c9bdae" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" is_pre_election: true
I20260811 14:30:26.762789 26617 raft_consensus.cc:2454] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 0.
I20260811 14:30:26.762822 26460 raft_consensus.cc:2454] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 0.
I20260811 14:30:26.762929 26697 leader_election.cc:302] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, 791d09c678fc4e2db789c1dab27c91ae; no voters:
I20260811 14:30:26.763007 26864 raft_consensus.cc:2781] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:26.763037 26864 raft_consensus.cc:482] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:26.763072 26864 raft_consensus.cc:3037] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.763939 26864 raft_consensus.cc:504] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.764104 26864 leader_election.cc:288] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 election: Requested vote from peers e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049), 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041)
I20260811 14:30:26.764320 26617 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba4f73be3b4c4cad77f5f702c9bdae" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1"
I20260811 14:30:26.764321 26460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a1ba4f73be3b4c4cad77f5f702c9bdae" candidate_uuid: "791d09c678fc4e2db789c1dab27c91ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08"
I20260811 14:30:26.764397 26460 raft_consensus.cc:3037] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.764397 26617 raft_consensus.cc:3037] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:26.765271 26460 raft_consensus.cc:2454] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 1.
I20260811 14:30:26.765290 26617 raft_consensus.cc:2454] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 791d09c678fc4e2db789c1dab27c91ae in term 1.
I20260811 14:30:26.765455 26697 leader_election.cc:302] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, 791d09c678fc4e2db789c1dab27c91ae; no voters:
I20260811 14:30:26.765537 26864 raft_consensus.cc:2781] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:26.765573 26864 raft_consensus.cc:686] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 1 LEADER]: Becoming Leader. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Running, Role: LEADER
I20260811 14:30:26.765620 26864 consensus_queue.cc:237] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:26.766170 26275 catalog_manager.cc:5495] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae reported cstate change: term changed from 0 to 1, leader changed from <none> to 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195). New cstate: current_term: 1 leader_uuid: "791d09c678fc4e2db789c1dab27c91ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } health_report { overall_health: HEALTHY } } }
I20260811 14:30:26.775913 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.773448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46836 (local address 127.23.186.195:38155)
0811 14:30:26.773513 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.773515 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.773522 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:26.773541 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.773542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.773551 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.773599 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:26.773743 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.774201 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.775269 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.775480 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.775547 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.775711 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.775741 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.775777 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.775831 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.775851 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:30:26.781284 26912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.778364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.23.186.195:51715)
0811 14:30:26.778419 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:26.778430 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:26.778476 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:26.778649 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:26.778650 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:26.778651 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:26.778689 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.778690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.779663 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.779664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.780272 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:26.780274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.780856 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.780857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:26.780898 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.780913 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.781126 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.781128 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:26.781164 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 14:30:26.781205 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 14:30:26.781363 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:26.778504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:51715 (local address 127.23.186.195:38155)
0811 14:30:26.778550 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:26.778551 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:26.778557 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:30:26.778572 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:26.778573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:26.778581 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:26.778620 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:26.778768 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.779578 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.780343 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:26.780639 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:26.780732 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:26.780986 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:26.781040 (+ 54us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:26.781120 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:26.781181 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 14:30:26.781242 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 14:30:26.783823 26460 raft_consensus.cc:1261] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Refusing update from remote peer 791d09c678fc4e2db789c1dab27c91ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:26.783819 26617 raft_consensus.cc:1261] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Refusing update from remote peer 791d09c678fc4e2db789c1dab27c91ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:26.784024 26885 consensus_queue.cc:1035] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:26.784155 26885 consensus_queue.cc:1035] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:26.786031 26891 mvcc.cc:205] Tried to move back new op lower bound from 7317334535305183232 to 7317334535232311296. Current Snapshot: MvccSnapshot[applied={T|T < 7317334535305183232}]
W20260811 14:30:26.789573 26890 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 14:30:28.311228 26276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:28.311339 26276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34680:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:28.312275 26276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Sending DeleteTablet for 3 replicas of tablet a1ba4f73be3b4c4cad77f5f702c9bdae
I20260811 14:30:28.312568 26434 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba4f73be3b4c4cad77f5f702c9bdae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:28 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:47690
I20260811 14:30:28.312597 26595 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba4f73be3b4c4cad77f5f702c9bdae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:28 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:49408
I20260811 14:30:28.312958 26947 tablet_replica.cc:330] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: stopping tablet replica
I20260811 14:30:28.313033 26947 raft_consensus.cc:2229] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:28.313020 26753 tablet_service.cc:1514] Processing DeleteTablet for tablet a1ba4f73be3b4c4cad77f5f702c9bdae with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:28 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260811 14:30:28.313086 26947 raft_consensus.cc:2258] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:28.313103 26948 tablet_replica.cc:330] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: stopping tablet replica
I20260811 14:30:28.313344 26948 raft_consensus.cc:2229] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:28.313360 26947 ts_tablet_manager.cc:1905] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:28.313401 26948 raft_consensus.cc:2258] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:28.313689 26948 ts_tablet_manager.cc:1905] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:28.315454 26947 ts_tablet_manager.cc:1918] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:30:28.315580 26947 log.cc:1196] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/a1ba4f73be3b4c4cad77f5f702c9bdae
I20260811 14:30:28.315810 26948 ts_tablet_manager.cc:1918] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:30:28.315862 26947 ts_tablet_manager.cc:1939] T a1ba4f73be3b4c4cad77f5f702c9bdae P 07e17b40b92b4e10a70876f5166c7dd1: Deleting consensus metadata
I20260811 14:30:28.315960 26948 log.cc:1196] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/a1ba4f73be3b4c4cad77f5f702c9bdae
I20260811 14:30:28.316247 26261 catalog_manager.cc:4841] TS 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041): tablet a1ba4f73be3b4c4cad77f5f702c9bdae (table test-table [id=c243e67b484f476e9e8b7819c36106f5]) successfully deleted
I20260811 14:30:28.316283 26948 ts_tablet_manager.cc:1939] T a1ba4f73be3b4c4cad77f5f702c9bdae P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting consensus metadata
I20260811 14:30:28.316684 26949 tablet_replica.cc:330] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: stopping tablet replica
I20260811 14:30:28.316738 26260 catalog_manager.cc:4841] TS e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049): tablet a1ba4f73be3b4c4cad77f5f702c9bdae (table test-table [id=c243e67b484f476e9e8b7819c36106f5]) successfully deleted
I20260811 14:30:28.316794 26949 raft_consensus.cc:2229] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:28.316926 26949 raft_consensus.cc:2258] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:28.317201 26949 ts_tablet_manager.cc:1905] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:28.318519 26949 ts_tablet_manager.cc:1918] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:30:28.318655 26949 log.cc:1196] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/a1ba4f73be3b4c4cad77f5f702c9bdae
I20260811 14:30:28.318965 26949 ts_tablet_manager.cc:1939] T a1ba4f73be3b4c4cad77f5f702c9bdae P 791d09c678fc4e2db789c1dab27c91ae: Deleting consensus metadata
I20260811 14:30:28.319317 26260 catalog_manager.cc:4841] TS 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155): tablet a1ba4f73be3b4c4cad77f5f702c9bdae (table test-table [id=c243e67b484f476e9e8b7819c36106f5]) successfully deleted
Aug 11 14:30:28 dist-test-slave-cbsf krb5kdc[26230](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:30:28 dist-test-slave-cbsf krb5kdc[26230](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458628, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:28.341033 26957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.332654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34702 (local address 127.23.186.254:41799)
0811 14:30:28.332870 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.332872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.332882 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:28.332933 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.332933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.332945 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.333009 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:28.333634 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.334155 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.334996 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.335196 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.335247 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.337993 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:28.337995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:28.337996 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:28.338004 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:28.339696 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:28.340109 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:28.340110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:28.340111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:28.340178 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:28.340418 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:28.340419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:28.340419 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:28.340544 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:28.340626 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.340833 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.340913 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":145,"threads_started":1}
I20260811 14:30:28.342612 26276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:28.342778 26276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:28.344358 26595 tablet_service.cc:1467] Processing CreateTablet for tablet ab6786aebe80406f851e6eee2a95ad2e (DEFAULT_TABLE table=test-table [id=f7f091a8ad664975b1722f98fff7bd9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:28.344367 26753 tablet_service.cc:1467] Processing CreateTablet for tablet ab6786aebe80406f851e6eee2a95ad2e (DEFAULT_TABLE table=test-table [id=f7f091a8ad664975b1722f98fff7bd9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:28.344360 26434 tablet_service.cc:1467] Processing CreateTablet for tablet ab6786aebe80406f851e6eee2a95ad2e (DEFAULT_TABLE table=test-table [id=f7f091a8ad664975b1722f98fff7bd9c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:28.344525 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6786aebe80406f851e6eee2a95ad2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:28.344540 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6786aebe80406f851e6eee2a95ad2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:28.344527 26753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ab6786aebe80406f851e6eee2a95ad2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:28.346006 26958 tablet_bootstrap.cc:492] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Bootstrap starting.
I20260811 14:30:28.346328 26959 tablet_bootstrap.cc:492] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Bootstrap starting.
I20260811 14:30:28.346347 26960 tablet_bootstrap.cc:492] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Bootstrap starting.
I20260811 14:30:28.346494 26958 tablet_bootstrap.cc:654] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:28.346722 26960 tablet_bootstrap.cc:654] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:28.346843 26959 tablet_bootstrap.cc:654] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:28.347460 26960 tablet_bootstrap.cc:492] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: No bootstrap required, opened a new log
I20260811 14:30:28.347469 26958 tablet_bootstrap.cc:492] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: No bootstrap required, opened a new log
I20260811 14:30:28.347536 26959 tablet_bootstrap.cc:492] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: No bootstrap required, opened a new log
I20260811 14:30:28.347532 26958 ts_tablet_manager.cc:1397] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:28.347533 26960 ts_tablet_manager.cc:1397] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:28.347586 26959 ts_tablet_manager.cc:1397] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:28.347667 26960 raft_consensus.cc:348] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.347724 26960 raft_consensus.cc:374] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:28.347666 26958 raft_consensus.cc:348] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.347738 26960 raft_consensus.cc:729] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Initialized, Role: FOLLOWER
I20260811 14:30:28.347748 26958 raft_consensus.cc:374] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:28.347774 26958 raft_consensus.cc:729] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Initialized, Role: FOLLOWER
I20260811 14:30:28.347800 26960 consensus_queue.cc:260] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.347827 26958 consensus_queue.cc:260] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.347903 26960 ts_tablet_manager.cc:1428] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:28.347889 26959 raft_consensus.cc:348] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.347971 26959 raft_consensus.cc:374] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:28.347993 26959 raft_consensus.cc:729] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e17b40b92b4e10a70876f5166c7dd1, State: Initialized, Role: FOLLOWER
I20260811 14:30:28.348001 26958 ts_tablet_manager.cc:1428] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:28.348047 26959 consensus_queue.cc:260] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.348217 26959 ts_tablet_manager.cc:1428] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:28.445600 26964 raft_consensus.cc:482] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:28.445720 26964 raft_consensus.cc:504] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.445981 26964 leader_election.cc:288] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041), 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:28.446250 26618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6786aebe80406f851e6eee2a95ad2e" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1" is_pre_election: true
I20260811 14:30:28.446256 26778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6786aebe80406f851e6eee2a95ad2e" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791d09c678fc4e2db789c1dab27c91ae" is_pre_election: true
I20260811 14:30:28.446376 26618 raft_consensus.cc:2454] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 0.
I20260811 14:30:28.446398 26778 raft_consensus.cc:2454] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 0.
I20260811 14:30:28.446552 26377 leader_election.cc:302] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, e4f4bb6c4de745f0b184ca7383dc0b08; no voters:
I20260811 14:30:28.446684 26964 raft_consensus.cc:2781] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:28.446766 26964 raft_consensus.cc:482] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:28.446795 26964 raft_consensus.cc:3037] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:28.447589 26964 raft_consensus.cc:504] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.447767 26964 leader_election.cc:288] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 election: Requested vote from peers 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041), 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:28.447999 26778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6786aebe80406f851e6eee2a95ad2e" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791d09c678fc4e2db789c1dab27c91ae"
I20260811 14:30:28.447999 26618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ab6786aebe80406f851e6eee2a95ad2e" candidate_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07e17b40b92b4e10a70876f5166c7dd1"
I20260811 14:30:28.448101 26778 raft_consensus.cc:3037] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:28.448101 26618 raft_consensus.cc:3037] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:28.449000 26778 raft_consensus.cc:2454] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 1.
I20260811 14:30:28.449000 26618 raft_consensus.cc:2454] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4f4bb6c4de745f0b184ca7383dc0b08 in term 1.
I20260811 14:30:28.449219 26376 leader_election.cc:302] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 791d09c678fc4e2db789c1dab27c91ae, e4f4bb6c4de745f0b184ca7383dc0b08; no voters:
I20260811 14:30:28.449318 26964 raft_consensus.cc:2781] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:28.449496 26964 raft_consensus.cc:686] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 LEADER]: Becoming Leader. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Running, Role: LEADER
I20260811 14:30:28.449545 26964 consensus_queue.cc:237] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.450309 26277 catalog_manager.cc:5495] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 reported cstate change: term changed from 0 to 1, leader changed from <none> to e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193). New cstate: current_term: 1 leader_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:28.470023 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.467527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47702 (local address 127.23.186.193:38049)
0811 14:30:28.467694 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.467696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.467704 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:28.467725 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.467725 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.467736 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.467799 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:28.467920 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.468419 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.469232 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.469510 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.469585 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.469744 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:28.469786 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:28.469838 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.469888 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.469926 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":107,"threads_started":1}
I20260811 14:30:28.473292 26778 raft_consensus.cc:1261] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Refusing update from remote peer e4f4bb6c4de745f0b184ca7383dc0b08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:28.473343 26618 raft_consensus.cc:1261] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Refusing update from remote peer e4f4bb6c4de745f0b184ca7383dc0b08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:28.473655 26964 consensus_queue.cc:1035] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:28.473790 26968 consensus_queue.cc:1035] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:28.485142 26275 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:28.485252 26275 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:28.485962 26275 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Sending DeleteTablet for 3 replicas of tablet ab6786aebe80406f851e6eee2a95ad2e
I20260811 14:30:28.486294 26434 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6786aebe80406f851e6eee2a95ad2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:28 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:47690
I20260811 14:30:28.486302 26595 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6786aebe80406f851e6eee2a95ad2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:28 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:49408
I20260811 14:30:28.486442 26947 tablet_replica.cc:330] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: stopping tablet replica
I20260811 14:30:28.486462 26948 tablet_replica.cc:330] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: stopping tablet replica
I20260811 14:30:28.486492 26947 raft_consensus.cc:2229] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:28.486531 26948 raft_consensus.cc:2229] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:28.486539 26947 raft_consensus.cc:2258] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:28.486594 26948 raft_consensus.cc:2258] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:28.486547 26275 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:28.486690 26275 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:28.486927 26948 ts_tablet_manager.cc:1905] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:28.486929 26753 tablet_service.cc:1514] Processing DeleteTablet for tablet ab6786aebe80406f851e6eee2a95ad2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:28 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260811 14:30:28.487001 26947 ts_tablet_manager.cc:1905] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:28.488092 26949 tablet_replica.cc:330] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: stopping tablet replica
I20260811 14:30:28.488200 26949 raft_consensus.cc:2229] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:28.488256 26949 raft_consensus.cc:2258] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:28.488466 26948 ts_tablet_manager.cc:1918] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:28.488561 26948 log.cc:1196] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/ab6786aebe80406f851e6eee2a95ad2e
I20260811 14:30:28.488624 26947 ts_tablet_manager.cc:1918] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:28.488739 26947 log.cc:1196] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/ab6786aebe80406f851e6eee2a95ad2e
I20260811 14:30:28.488853 26948 ts_tablet_manager.cc:1939] T ab6786aebe80406f851e6eee2a95ad2e P e4f4bb6c4de745f0b184ca7383dc0b08: Deleting consensus metadata
I20260811 14:30:28.489009 26947 ts_tablet_manager.cc:1939] T ab6786aebe80406f851e6eee2a95ad2e P 07e17b40b92b4e10a70876f5166c7dd1: Deleting consensus metadata
I20260811 14:30:28.489202 26260 catalog_manager.cc:4841] TS e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049): tablet ab6786aebe80406f851e6eee2a95ad2e (table test-table [id=f7f091a8ad664975b1722f98fff7bd9c]) successfully deleted
I20260811 14:30:28.489233 26434 tablet_service.cc:1467] Processing CreateTablet for tablet 9eefd229c7ea45b49625db86334ac933 (DEFAULT_TABLE table=test-table [id=624c087fdbe044b79126e849ecfec45d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:28.489360 26434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eefd229c7ea45b49625db86334ac933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:28.489363 26261 catalog_manager.cc:4841] TS 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041): tablet ab6786aebe80406f851e6eee2a95ad2e (table test-table [id=f7f091a8ad664975b1722f98fff7bd9c]) successfully deleted
I20260811 14:30:28.489382 26595 tablet_service.cc:1467] Processing CreateTablet for tablet 9eefd229c7ea45b49625db86334ac933 (DEFAULT_TABLE table=test-table [id=624c087fdbe044b79126e849ecfec45d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:28.489383 26753 tablet_service.cc:1467] Processing CreateTablet for tablet 9eefd229c7ea45b49625db86334ac933 (DEFAULT_TABLE table=test-table [id=624c087fdbe044b79126e849ecfec45d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:28.489481 26595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eefd229c7ea45b49625db86334ac933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:28.489481 26753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eefd229c7ea45b49625db86334ac933. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:28.490367 26958 tablet_bootstrap.cc:492] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08: Bootstrap starting.
I20260811 14:30:28.490693 26959 tablet_bootstrap.cc:492] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Bootstrap starting.
I20260811 14:30:28.490758 26958 tablet_bootstrap.cc:654] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:28.490928 26960 tablet_bootstrap.cc:492] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Bootstrap starting.
I20260811 14:30:28.491061 26949 ts_tablet_manager.cc:1905] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:28.491144 26959 tablet_bootstrap.cc:654] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:28.491266 26960 tablet_bootstrap.cc:654] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:28.491724 26959 tablet_bootstrap.cc:492] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: No bootstrap required, opened a new log
I20260811 14:30:28.491796 26959 ts_tablet_manager.cc:1397] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:28.491916 26960 tablet_bootstrap.cc:492] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: No bootstrap required, opened a new log
I20260811 14:30:28.491976 26960 ts_tablet_manager.cc:1397] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:28.491976 26959 raft_consensus.cc:348] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.492053 26959 raft_consensus.cc:374] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:28.492077 26959 raft_consensus.cc:729] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e17b40b92b4e10a70876f5166c7dd1, State: Initialized, Role: FOLLOWER
I20260811 14:30:28.492105 26960 raft_consensus.cc:348] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.492137 26959 consensus_queue.cc:260] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.492170 26960 raft_consensus.cc:374] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:28.492189 26960 raft_consensus.cc:729] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791d09c678fc4e2db789c1dab27c91ae, State: Initialized, Role: FOLLOWER
I20260811 14:30:28.492218 26960 consensus_queue.cc:260] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.492233 26959 ts_tablet_manager.cc:1428] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:28.492306 26960 ts_tablet_manager.cc:1428] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:28.492633 26949 ts_tablet_manager.cc:1918] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:28.492720 26949 log.cc:1196] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/ab6786aebe80406f851e6eee2a95ad2e
I20260811 14:30:28.492954 26949 ts_tablet_manager.cc:1939] T ab6786aebe80406f851e6eee2a95ad2e P 791d09c678fc4e2db789c1dab27c91ae: Deleting consensus metadata
I20260811 14:30:28.493410 26260 catalog_manager.cc:4841] TS 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155): tablet ab6786aebe80406f851e6eee2a95ad2e (table test-table [id=f7f091a8ad664975b1722f98fff7bd9c]) successfully deleted
I20260811 14:30:28.493719 26958 tablet_bootstrap.cc:492] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08: No bootstrap required, opened a new log
I20260811 14:30:28.493793 26958 ts_tablet_manager.cc:1397] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 14:30:28.493918 26958 raft_consensus.cc:348] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.494004 26958 raft_consensus.cc:374] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:28.494040 26958 raft_consensus.cc:729] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4f4bb6c4de745f0b184ca7383dc0b08, State: Initialized, Role: FOLLOWER
I20260811 14:30:28.494091 26958 consensus_queue.cc:260] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.494189 26958 ts_tablet_manager.cc:1428] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:28.516537 26967 raft_consensus.cc:482] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:28.516642 26967 raft_consensus.cc:504] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.517071 26967 leader_election.cc:288] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049), 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:28.521502 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.517633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:42151 (local address 127.23.186.195:38155)
0811 14:30:28.517809 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.517811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.517818 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:28.517847 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.517847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.517865 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.517955 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:28.518112 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.518813 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.520444 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.521065 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.521134 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.521144 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.521372 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.521424 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260811 14:30:28.521502 26971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.517372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:42227 (local address 127.23.186.193:38049)
0811 14:30:28.517476 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.517479 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.517559 (+ 80us) server_negotiation.cc:407] Connection header received
0811 14:30:28.517670 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.517671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.517682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.517717 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:28.517961 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.518778 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.520445 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.521043 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.521133 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.521144 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.521374 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.521417 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 14:30:28.521574 26982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.517527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.23.186.194:42151)
0811 14:30:28.517676 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:28.517685 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:28.517754 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:28.517980 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:28.517981 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:28.517983 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:28.518037 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.518038 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.518896 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.518897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.520333 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.520336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.521172 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.521174 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.521275 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.521280 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:28.521353 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:30:28.521425 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":90,"thread_start_us":90,"threads_started":1}
I20260811 14:30:28.521574 26980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.517280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38049 (local address 127.23.186.194:42227)
0811 14:30:28.517471 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:28.517485 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:28.517566 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:28.517799 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:28.517800 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:28.517802 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:28.517877 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.517878 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.518921 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.518922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.520333 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.520336 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.521173 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.521174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.521275 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.521282 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:30:28.521353 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:30:28.521425 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":78,"thread_start_us":116,"threads_started":1}
I20260811 14:30:28.521970 26778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eefd229c7ea45b49625db86334ac933" candidate_uuid: "07e17b40b92b4e10a70876f5166c7dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791d09c678fc4e2db789c1dab27c91ae" is_pre_election: true
I20260811 14:30:28.522085 26778 raft_consensus.cc:2454] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07e17b40b92b4e10a70876f5166c7dd1 in term 0.
I20260811 14:30:28.522221 26459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eefd229c7ea45b49625db86334ac933" candidate_uuid: "07e17b40b92b4e10a70876f5166c7dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" is_pre_election: true
I20260811 14:30:28.522327 26459 raft_consensus.cc:2454] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07e17b40b92b4e10a70876f5166c7dd1 in term 0.
I20260811 14:30:28.522286 26538 leader_election.cc:302] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, 791d09c678fc4e2db789c1dab27c91ae; no voters:
I20260811 14:30:28.522506 26967 raft_consensus.cc:2781] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:28.522567 26967 raft_consensus.cc:482] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:28.522584 26967 raft_consensus.cc:3037] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:28.523392 26967 raft_consensus.cc:504] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.523547 26967 leader_election.cc:288] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049), 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155)
I20260811 14:30:28.523748 26459 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eefd229c7ea45b49625db86334ac933" candidate_uuid: "07e17b40b92b4e10a70876f5166c7dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08"
I20260811 14:30:28.523773 26778 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eefd229c7ea45b49625db86334ac933" candidate_uuid: "07e17b40b92b4e10a70876f5166c7dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "791d09c678fc4e2db789c1dab27c91ae"
I20260811 14:30:28.523844 26459 raft_consensus.cc:3037] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:28.523841 26778 raft_consensus.cc:3037] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:28.524560 26459 raft_consensus.cc:2454] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07e17b40b92b4e10a70876f5166c7dd1 in term 1.
I20260811 14:30:28.524601 26778 raft_consensus.cc:2454] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07e17b40b92b4e10a70876f5166c7dd1 in term 1.
I20260811 14:30:28.524788 26538 leader_election.cc:302] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07e17b40b92b4e10a70876f5166c7dd1, e4f4bb6c4de745f0b184ca7383dc0b08; no voters:
I20260811 14:30:28.524923 26967 raft_consensus.cc:2781] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:28.525120 26967 raft_consensus.cc:686] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 LEADER]: Becoming Leader. State: Replica: 07e17b40b92b4e10a70876f5166c7dd1, State: Running, Role: LEADER
I20260811 14:30:28.525206 26967 consensus_queue.cc:237] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } }
I20260811 14:30:28.526052 26275 catalog_manager.cc:5495] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "07e17b40b92b4e10a70876f5166c7dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07e17b40b92b4e10a70876f5166c7dd1" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:28.537981 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.535269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49412 (local address 127.23.186.194:44041)
0811 14:30:28.535493 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.535495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.535501 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:30:28.535544 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.535545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.535568 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.535638 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:28.535764 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.536267 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.537056 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.537281 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.537340 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.537558 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:28.537683 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:28.537756 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.537805 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.537828 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":160,"threads_started":1}
I20260811 14:30:28.543411 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.540680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:38155 (local address 127.23.186.194:40921)
0811 14:30:28.540944 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:28.540958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:28.541020 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:28.541168 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:28.541169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:28.541170 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:28.541245 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.541251 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.541891 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.541892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.542504 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.542505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.542914 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.542916 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.542982 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.542998 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:28.543213 (+ 215us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:28.543216 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:28.543268 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:30:28.543325 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":151,"threads_started":1}
I20260811 14:30:28.543387 26984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.540882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:40921 (local address 127.23.186.195:38155)
0811 14:30:28.540952 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.540954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.541003 (+ 49us) server_negotiation.cc:407] Connection header received
0811 14:30:28.541074 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.541076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.541089 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.541165 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:28.541320 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.541803 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.542599 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.542808 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.542889 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.543105 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:28.543146 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:28.543210 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.543273 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.543300 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:30:28.545819 26459 raft_consensus.cc:1261] T 9eefd229c7ea45b49625db86334ac933 P e4f4bb6c4de745f0b184ca7383dc0b08 [term 1 FOLLOWER]: Refusing update from remote peer 07e17b40b92b4e10a70876f5166c7dd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:28.545804 26778 raft_consensus.cc:1261] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Refusing update from remote peer 07e17b40b92b4e10a70876f5166c7dd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:28.546381 26985 consensus_queue.cc:1035] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "791d09c678fc4e2db789c1dab27c91ae" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 38155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:28.546607 26967 consensus_queue.cc:1035] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4f4bb6c4de745f0b184ca7383dc0b08" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 38049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:28.555106 26988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.552475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:39625 (local address 127.23.186.194:44041)
0811 14:30:28.552516 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:28.552518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:28.552649 (+ 131us) server_negotiation.cc:407] Connection header received
0811 14:30:28.552822 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:28.552823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:28.552841 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:28.552912 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:28.553091 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.553585 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.554450 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.554700 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.554761 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.554914 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:28.554944 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:28.554985 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:28.555022 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 14:30:28.555048 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:30:28.555122 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:28.552415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:44041 (local address 127.23.186.195:39625)
0811 14:30:28.552584 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:28.552597 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:28.552657 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:28.552917 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:28.552918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:28.552920 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:28.553000 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.553002 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.553675 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.553676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.554362 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:28.554365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:28.554774 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:28.554774 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:28.554840 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:28.554846 (+ 6us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:28.554983 (+ 137us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:28.554984 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:30:28.555011 (+ 27us) client_negotiation.cc:240] Negotiation successful
0811 14:30:28.555051 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":98,"threads_started":1}
W20260811 14:30:28.555586 26930 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 14:30:30.081183 26277 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:30.081305 26277 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:30.082149 26277 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4d8c08a8169e497c8543a1c3cdb914d1: Sending DeleteTablet for 3 replicas of tablet 9eefd229c7ea45b49625db86334ac933
I20260811 14:30:30.082455 26595 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eefd229c7ea45b49625db86334ac933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:30 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:49408
I20260811 14:30:30.082491 26434 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eefd229c7ea45b49625db86334ac933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:30 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:47690
I20260811 14:30:30.082537 26753 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eefd229c7ea45b49625db86334ac933 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:30 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46818
I20260811 14:30:30.082635 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 26360
I20260811 14:30:30.084022 27027 tablet_replica.cc:330] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: stopping tablet replica
I20260811 14:30:30.084086 27027 raft_consensus.cc:2229] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:30.084126 27027 raft_consensus.cc:2258] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:30.085410 27027 ts_tablet_manager.cc:1905] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:30.087868 27027 ts_tablet_manager.cc:1918] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:30:30.087952 27027 log.cc:1196] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/9eefd229c7ea45b49625db86334ac933
I20260811 14:30:30.088245 27027 ts_tablet_manager.cc:1939] T 9eefd229c7ea45b49625db86334ac933 P 791d09c678fc4e2db789c1dab27c91ae: Deleting consensus metadata
I20260811 14:30:30.088753 26260 catalog_manager.cc:4841] TS 791d09c678fc4e2db789c1dab27c91ae (127.23.186.195:38155): tablet 9eefd229c7ea45b49625db86334ac933 (table test-table [id=624c087fdbe044b79126e849ecfec45d]) successfully deleted
I20260811 14:30:30.092927 27025 tablet_replica.cc:330] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: stopping tablet replica
I20260811 14:30:30.093006 27025 raft_consensus.cc:2229] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:30.093135 27025 raft_consensus.cc:2258] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:30.093694 27025 ts_tablet_manager.cc:1905] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:30.096218 27025 ts_tablet_manager.cc:1918] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 14:30:30.096318 27025 log.cc:1196] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/9eefd229c7ea45b49625db86334ac933
I20260811 14:30:30.096614 27025 ts_tablet_manager.cc:1939] T 9eefd229c7ea45b49625db86334ac933 P 07e17b40b92b4e10a70876f5166c7dd1: Deleting consensus metadata
I20260811 14:30:30.097074 26261 catalog_manager.cc:4841] TS 07e17b40b92b4e10a70876f5166c7dd1 (127.23.186.194:44041): tablet 9eefd229c7ea45b49625db86334ac933 (table test-table [id=624c087fdbe044b79126e849ecfec45d]) successfully deleted
W20260811 14:30:30.100041 26260 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:38049 (error 108)
I20260811 14:30:30.100323 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 26521
I20260811 14:30:30.102192 27030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:30.101881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:38049 (local address 127.0.0.1:47708)
0811 14:30:30.102058 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:30.102134 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:38049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
W20260811 14:30:30.102342 26260 catalog_manager.cc:4584] TS e4f4bb6c4de745f0b184ca7383dc0b08 (127.23.186.193:38049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9eefd229c7ea45b49625db86334ac933: Network error: Client connection negotiation failed: client connection to 127.23.186.193:38049: connect: Connection refused (error 111)
I20260811 14:30:30.110651 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 26679
I20260811 14:30:30.121289 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 26243
2026-08-11T14:30:30Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6120 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:30 dist-test-slave-cbsf krb5kdc[27033](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:30 dist-test-slave-cbsf krb5kdc[27033](info): set up 2 sockets
Aug 11 14:30:30 dist-test-slave-cbsf krb5kdc[27033](info): commencing operation
krb5kdc: starting...
W20260811 14:30:31.261289 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:31.396800 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39831
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39831
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:31.484217 27048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:31.484505 27048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:31.484545 27048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:31.485824 27048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:31.485909 27048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:31.485934 27048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:31.485955 27048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:31.485987 27048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:31.487600 27048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:31.487920 27048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:31.488179 27048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:31.493741 27056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:31.493743 27054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:31.493798 27048 server_base.cc:1034] running on GCE node
W20260811 14:30:31.493745 27053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:31.494215 27048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:31.494551 27048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:31.495723 27048 hybrid_clock.cc:648] HybridClock initialized: now 1786458631495699 us; error 39 us; skew 500 ppm
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:31.503479 27048 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:31.504034 27048 webserver.cc:460] Webserver started at http://127.23.186.254:46867/ using document root <none> and password file <none>
I20260811 14:30:31.504302 27048 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:31.504356 27048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:31.504549 27048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:31.505805 27048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "5c9623ba1bad43d2972c5de1905c8cb7"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "0290214a8dbea4da8b8efd7f9661f388"
server_key_iv: "5d0fc0b400ac890ad5633a9edb5e7d9c"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.506213 27048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "5c9623ba1bad43d2972c5de1905c8cb7"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "0290214a8dbea4da8b8efd7f9661f388"
server_key_iv: "5d0fc0b400ac890ad5633a9edb5e7d9c"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.507726 27048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:31.508579 27063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.508843 27048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:31.508952 27048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "5c9623ba1bad43d2972c5de1905c8cb7"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "0290214a8dbea4da8b8efd7f9661f388"
server_key_iv: "5d0fc0b400ac890ad5633a9edb5e7d9c"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.509039 27048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:31.522329 27048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:31.524417 27048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:31.524596 27048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:31.528789 27048 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:39831
I20260811 14:30:31.528860 27124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:39831 every 8 connection(s)
I20260811 14:30:31.529243 27048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:31.530251 27125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:31.532848 27125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7: Bootstrap starting.
I20260811 14:30:31.533331 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27048
I20260811 14:30:31.533439 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:31.533651 24299 external_mini_cluster.cc:1428] Setting key 28ba0b60a7948ef0a1a4d755bc4bd9a2
I20260811 14:30:31.533712 27125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:31.534050 27125 log.cc:824] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:31.534986 27125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7: No bootstrap required, opened a new log
I20260811 14:30:31.537699 27125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39831 } }
I20260811 14:30:31.537933 27125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:31.537987 27125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9623ba1bad43d2972c5de1905c8cb7, State: Initialized, Role: FOLLOWER
I20260811 14:30:31.538163 27125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39831 } }
I20260811 14:30:31.538268 27125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:31.538316 27125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:31.538368 27125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:31.539233 27125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39831 } }
I20260811 14:30:31.539366 27125 leader_election.cc:302] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c9623ba1bad43d2972c5de1905c8cb7; no voters:
I20260811 14:30:31.539601 27125 leader_election.cc:288] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:31.539675 27131 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:31.539872 27131 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [term 1 LEADER]: Becoming Leader. State: Replica: 5c9623ba1bad43d2972c5de1905c8cb7, State: Running, Role: LEADER
I20260811 14:30:31.539937 27125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:31.540073 27131 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39831 } }
I20260811 14:30:31.540506 27133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c9623ba1bad43d2972c5de1905c8cb7. Latest consensus state: current_term: 1 leader_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39831 } } }
I20260811 14:30:31.540607 27133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:31.541010 27138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:31.541594 27132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c9623ba1bad43d2972c5de1905c8cb7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39831 } } }
I20260811 14:30:31.541765 27132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:31.541886 27138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:31.544051 27138 catalog_manager.cc:1347] Generated new cluster ID: cab10d9ba9ea47e7ad9bcf49de6c38a2
I20260811 14:30:31.544118 27138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:31.546839 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:31.534249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33806 (local address 127.23.186.254:39831)
0811 14:30:31.534436 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:31.534445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:31.534476 (+ 31us) server_negotiation.cc:407] Connection header received
0811 14:30:31.535174 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:31.535184 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:31.535254 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:31.535418 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:31.536046 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.537650 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.538677 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.539350 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.539471 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:31.542697 (+ 3226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:31.542703 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:31.542703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:31.542717 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:31.544970 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:31.545425 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:31.545429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:31.545429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:31.545552 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:31.545771 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:31.545771 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:31.545772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:31.546148 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:31.546328 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:31.546426 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 14:30:31.546596 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260811 14:30:31.564738 27138 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:31.565388 27138 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:31.567616 27138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c9623ba1bad43d2972c5de1905c8cb7: Generated new TSK 0
I20260811 14:30:31.567827 27138 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:31.604045 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:39831
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:31.691533 27153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:31.691803 27153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:31.691849 27153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:31.693203 27153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:31.693274 27153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:31.693321 27153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:31.694911 27153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:39831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:31.695241 27153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:31.695530 27153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:30:31.701226 27153 server_base.cc:1034] running on GCE node
W20260811 14:30:31.701195 27158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:31.701197 27159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:31.701195 27162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:31.701727 27153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:31.702018 27153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:31.703223 27153 hybrid_clock.cc:648] HybridClock initialized: now 1786458631703208 us; error 56 us; skew 500 ppm
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:31.706167 27153 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:31.706696 27153 webserver.cc:460] Webserver started at http://127.23.186.193:44221/ using document root <none> and password file <none>
I20260811 14:30:31.706924 27153 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:31.706979 27153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:31.707082 27153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:31.708177 27153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "ea4ef6db87284829bbf12b70871a0b09"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "21d4b938050f9970f4b3e243e4e6d576"
server_key_iv: "639c39e17fb7a2371d9d20d11e81db79"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.708508 27153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "ea4ef6db87284829bbf12b70871a0b09"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "21d4b938050f9970f4b3e243e4e6d576"
server_key_iv: "639c39e17fb7a2371d9d20d11e81db79"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.710055 27153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:31.710916 27170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.711156 27153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:31.711259 27153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "ea4ef6db87284829bbf12b70871a0b09"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "21d4b938050f9970f4b3e243e4e6d576"
server_key_iv: "639c39e17fb7a2371d9d20d11e81db79"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.711323 27153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:31.731662 27153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:31.734519 27153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:31.734664 27153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:31.735014 27153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:31.735406 27153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:31.735450 27153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.735482 27153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:31.735505 27153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.742201 27153 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:40773
I20260811 14:30:31.742512 27298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:40773 every 8 connection(s)
I20260811 14:30:31.742700 27153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:31.751191 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27153
I20260811 14:30:31.751313 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:31.751534 24299 external_mini_cluster.cc:1428] Setting key 0bfe93122f25b35ade99c869ceccff5c
I20260811 14:30:31.754045 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:31.743822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:42635 (local address 127.23.186.254:39831)
0811 14:30:31.743901 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:31.743903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:31.744144 (+ 241us) server_negotiation.cc:407] Connection header received
0811 14:30:31.744639 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:31.744640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:31.744651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:31.744724 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:31.745928 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.746453 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.747540 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.747746 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.747815 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:31.750346 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:31.750349 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:31.750349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:31.750361 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:31.752398 (+ 2037us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:31.752942 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:31.752943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:31.752944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:31.753019 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:31.753305 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:31.753306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:31.753307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:31.753443 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:31.753526 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:31.753844 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 14:30:31.753949 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 14:30:31.754274 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:31.743794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:39831 (local address 127.23.186.193:42635)
0811 14:30:31.744002 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:31.744049 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:31.744544 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:31.744959 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:31.744964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:31.745613 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:31.745832 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:31.745836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.746551 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.746552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:31.747425 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:31.747427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.747830 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.747831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:31.747942 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:31.748489 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:31.748499 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:31.750174 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:31.752495 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:31.752506 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:31.752509 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:31.752861 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:31.753091 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:31.753092 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:31.753092 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:31.753228 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:31.753520 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:31.753528 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:31.753699 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 14:30:31.753808 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 14:30:31.753993 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
I20260811 14:30:31.754966 27299 heartbeater.cc:347] Connected to a master server at 127.23.186.254:39831
I20260811 14:30:31.755087 27299 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:31.755319 27299 heartbeater.cc:511] Master 127.23.186.254:39831 requested a full tablet report, sending...
I20260811 14:30:31.755975 27084 ts_manager.cc:194] Registered new tserver with Master: ea4ef6db87284829bbf12b70871a0b09 (127.23.186.193:40773)
I20260811 14:30:31.756682 27084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:42635
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:31.783444 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:39831
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:31.871415 27305 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:31.871707 27305 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:31.871750 27305 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:31.873732 27305 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:31.873831 27305 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:31.873881 27305 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:31.875898 27305 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:39831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27305
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:31.876221 27305 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:31.876503 27305 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:31.883837 27312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:31.884469 27311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:31.885249 27315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:31.885581 27305 server_base.cc:1034] running on GCE node
I20260811 14:30:31.885838 27305 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:31.886123 27305 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:31.887265 27305 hybrid_clock.cc:648] HybridClock initialized: now 1786458631887235 us; error 47 us; skew 500 ppm
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:31.890290 27305 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:31.890765 27305 webserver.cc:460] Webserver started at http://127.23.186.194:41171/ using document root <none> and password file <none>
I20260811 14:30:31.891055 27305 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:31.891111 27305 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:31.891258 27305 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:31.892453 27305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "13116edc0adc46db918ecd2ce52c1bcb"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "a702d26d15ad1ac338d1f8a9980eb983"
server_key_iv: "e4e7c051c6332cccba4abdbebc348b08"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.892885 27305 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "13116edc0adc46db918ecd2ce52c1bcb"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "a702d26d15ad1ac338d1f8a9980eb983"
server_key_iv: "e4e7c051c6332cccba4abdbebc348b08"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.894415 27305 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:31.895319 27322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.895618 27305 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:31.895742 27305 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "13116edc0adc46db918ecd2ce52c1bcb"
format_stamp: "Formatted at 2026-08-11 14:30:31 on dist-test-slave-cbsf"
server_key: "a702d26d15ad1ac338d1f8a9980eb983"
server_key_iv: "e4e7c051c6332cccba4abdbebc348b08"
server_key_version: "encryptionkey@0"
I20260811 14:30:31.895830 27305 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:31.913977 27305 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:31.916338 27305 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:31.916507 27305 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:31.916895 27305 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:31.917281 27305 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:31.917317 27305 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.917362 27305 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:31.917388 27305 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:31.924376 27305 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:45429
I20260811 14:30:31.924484 27448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:45429 every 8 connection(s)
I20260811 14:30:31.924865 27305 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:31.930361 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27305
I20260811 14:30:31.930513 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:31.930792 24299 external_mini_cluster.cc:1428] Setting key 8d28f8473f8730e912fbd283b22493a9
Aug 11 14:30:31 dist-test-slave-cbsf krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458631, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:31.936148 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:31.925700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:45787 (local address 127.23.186.254:39831)
0811 14:30:31.925770 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:31.925772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:31.925982 (+ 210us) server_negotiation.cc:407] Connection header received
0811 14:30:31.926545 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:31.926546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:31.926559 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:31.926613 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:31.927680 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.928222 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.929355 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.929575 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.929665 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:31.932585 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:31.932587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:31.932587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:31.932596 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:31.934323 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:31.934943 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:31.934944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:31.934945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:31.935003 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:31.935344 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:31.935345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:31.935345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:31.935532 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:31.935606 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:31.935903 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 14:30:31.936003 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:30:31.936196 27452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:31.925683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:39831 (local address 127.23.186.194:45787)
0811 14:30:31.925843 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:31.925884 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:31.926422 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:31.926691 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:31.926696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:31.927336 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:31.927571 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:31.927576 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.928329 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.928330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:31.929249 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:31.929252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:31.929679 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:31.929681 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:31.929813 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:31.930444 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:31.930455 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:31.932461 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:31.934427 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:31.934438 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:31.934441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:31.934852 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:31.935112 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:31.935112 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:31.935113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:31.935270 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:31.935614 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:31.935619 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:31.935761 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 14:30:31.935853 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:30:31.936001 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260811 14:30:31.936978 27449 heartbeater.cc:347] Connected to a master server at 127.23.186.254:39831
I20260811 14:30:31.937108 27449 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:31.937361 27449 heartbeater.cc:511] Master 127.23.186.254:39831 requested a full tablet report, sending...
I20260811 14:30:31.937855 27084 ts_manager.cc:194] Registered new tserver with Master: 13116edc0adc46db918ecd2ce52c1bcb (127.23.186.194:45429)
I20260811 14:30:31.938272 27084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:45787
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:31.963683 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:39831
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:32.052552 27456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:32.052875 27456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:32.052923 27456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:32.054206 27456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:32.054279 27456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:32.054329 27456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:32.055969 27456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:39831
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:32.056265 27456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:32.056548 27456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:32.062106 27462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:32.062098 27463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:32.062242 27456 server_base.cc:1034] running on GCE node
W20260811 14:30:32.062098 27465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:32.062628 27456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:32.062956 27456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:32.064148 27456 hybrid_clock.cc:648] HybridClock initialized: now 1786458632064133 us; error 49 us; skew 500 ppm
Aug 11 14:30:32 dist-test-slave-cbsf krb5kdc[27033](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458632, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:32.067020 27456 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:32.067543 27456 webserver.cc:460] Webserver started at http://127.23.186.195:33683/ using document root <none> and password file <none>
I20260811 14:30:32.067875 27456 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:32.067951 27456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:32.068132 27456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:32.069335 27456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "c5cbafc9831c4ca8a9013685c39ff890"
format_stamp: "Formatted at 2026-08-11 14:30:32 on dist-test-slave-cbsf"
server_key: "3dceb233f328de2bd8da0399ef1d7a80"
server_key_iv: "84a25023795f981ce22e0dfe97672d7c"
server_key_version: "encryptionkey@0"
I20260811 14:30:32.069741 27456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "c5cbafc9831c4ca8a9013685c39ff890"
format_stamp: "Formatted at 2026-08-11 14:30:32 on dist-test-slave-cbsf"
server_key: "3dceb233f328de2bd8da0399ef1d7a80"
server_key_iv: "84a25023795f981ce22e0dfe97672d7c"
server_key_version: "encryptionkey@0"
I20260811 14:30:32.071197 27456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:32.072077 27474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:32.072309 27456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:32.072419 27456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "c5cbafc9831c4ca8a9013685c39ff890"
format_stamp: "Formatted at 2026-08-11 14:30:32 on dist-test-slave-cbsf"
server_key: "3dceb233f328de2bd8da0399ef1d7a80"
server_key_iv: "84a25023795f981ce22e0dfe97672d7c"
server_key_version: "encryptionkey@0"
I20260811 14:30:32.072508 27456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:32.093048 27456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:32.095247 27456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:32.095439 27456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:32.095719 27456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:32.096098 27456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:32.096156 27456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:32.096217 27456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:32.096268 27456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:32.103106 27456 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:44047
I20260811 14:30:32.103247 27612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:44047 every 8 connection(s)
I20260811 14:30:32.103575 27456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:32.110299 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27456
I20260811 14:30:32.110410 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
Aug 11 14:30:32 dist-test-slave-cbsf krb5kdc[27033](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458632, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:32.110601 24299 external_mini_cluster.cc:1428] Setting key 17e49819d902f401f2f029b3c53750aa
I20260811 14:30:32.114511 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:32.104455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:37885 (local address 127.23.186.254:39831)
0811 14:30:32.104536 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:32.104539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:32.104823 (+ 284us) server_negotiation.cc:407] Connection header received
0811 14:30:32.105376 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:32.105378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:32.105390 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:32.105450 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:32.106504 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:32.107034 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:32.108069 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:32.108276 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:32.108351 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:32.111149 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:32.111150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:32.111151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:32.111159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:32.112899 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:32.113358 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:32.113360 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:32.113360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:32.113425 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:32.113724 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:32.113725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:32.113725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:32.113856 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:32.113933 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:32.114317 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 14:30:32.114412 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:30:32.114737 27615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:32.104421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:39831 (local address 127.23.186.195:37885)
0811 14:30:32.104644 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:32.104686 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:32.105260 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:32.105546 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:32.105550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:32.106181 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:32.106396 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:32.106400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:32.107129 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:32.107129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:32.107985 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:32.107987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:32.108389 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:32.108390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:32.108508 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:32.109181 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:32.109191 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:32.111060 (+ 1869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:32.112991 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:32.113001 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:32.113004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:32.113274 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:32.113499 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:32.113500 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:32.113501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:32.113639 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:32.113935 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:32.113943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:32.114167 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 14:30:32.114283 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 14:30:32.114470 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":120,"threads_started":1}
I20260811 14:30:32.115264 27613 heartbeater.cc:347] Connected to a master server at 127.23.186.254:39831
I20260811 14:30:32.115360 27613 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:32.115540 27613 heartbeater.cc:511] Master 127.23.186.254:39831 requested a full tablet report, sending...
I20260811 14:30:32.115916 27084 ts_manager.cc:194] Registered new tserver with Master: c5cbafc9831c4ca8a9013685c39ff890 (127.23.186.195:44047)
I20260811 14:30:32.116274 27084 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:37885
I20260811 14:30:32.122784 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 14:30:32.137473 27130 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:32.136354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.23.186.254:39831)
0811 14:30:32.136427 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:32.136429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:32.136461 (+ 32us) server_negotiation.cc:407] Connection header received
0811 14:30:32.136528 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:32.136528 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:32.136541 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:32.136609 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:32.137387 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33820: BlockingRecv error: recv got EOF from 127.0.0.1:33820 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:30:32.138450 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27153
I20260811 14:30:32.148552 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27305
I20260811 14:30:32.159406 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27456
I20260811 14:30:32.173884 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27048
2026-08-11T14:30:32Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2049 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:32 dist-test-slave-cbsf krb5kdc[27625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:32 dist-test-slave-cbsf krb5kdc[27625](info): set up 2 sockets
Aug 11 14:30:32 dist-test-slave-cbsf krb5kdc[27625](info): commencing operation
krb5kdc: starting...
W20260811 14:30:33.299969 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:33 dist-test-slave-cbsf krb5kdc[27625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:33.433612 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39807
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:33.522174 27641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:33.522452 27641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:33.522492 27641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:33.523756 27641 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:33.523835 27641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:33.523862 27641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:33.523895 27641 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:33.523917 27641 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:33.525660 27641 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:33.525979 27641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:33.526242 27641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:33.531666 27650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:33.531677 27652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:33.531685 27649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:33.531978 27641 server_base.cc:1034] running on GCE node
I20260811 14:30:33.532254 27641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:33.532562 27641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:33.533746 27641 hybrid_clock.cc:648] HybridClock initialized: now 1786458633533744 us; error 49 us; skew 500 ppm
Aug 11 14:30:33 dist-test-slave-cbsf krb5kdc[27625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458633, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:33.536967 27641 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:33.537640 27641 webserver.cc:460] Webserver started at http://127.23.186.254:36147/ using document root <none> and password file <none>
I20260811 14:30:33.537936 27641 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:33.538033 27641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:33.538226 27641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:33.539438 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "7506bcba91074765a06d585349e23ef0"
format_stamp: "Formatted at 2026-08-11 14:30:33 on dist-test-slave-cbsf"
server_key: "970df80f96d2b0c429cb2803b5b0ed53"
server_key_iv: "87a88e7a5f410f1a7f879a1186290bc2"
server_key_version: "encryptionkey@0"
I20260811 14:30:33.539834 27641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "7506bcba91074765a06d585349e23ef0"
format_stamp: "Formatted at 2026-08-11 14:30:33 on dist-test-slave-cbsf"
server_key: "970df80f96d2b0c429cb2803b5b0ed53"
server_key_iv: "87a88e7a5f410f1a7f879a1186290bc2"
server_key_version: "encryptionkey@0"
I20260811 14:30:33.541402 27641 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:30:33.542258 27659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:33.542482 27641 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:33.542601 27641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "7506bcba91074765a06d585349e23ef0"
format_stamp: "Formatted at 2026-08-11 14:30:33 on dist-test-slave-cbsf"
server_key: "970df80f96d2b0c429cb2803b5b0ed53"
server_key_iv: "87a88e7a5f410f1a7f879a1186290bc2"
server_key_version: "encryptionkey@0"
I20260811 14:30:33.542682 27641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:33.570405 27641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:33.572494 27641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:33.572664 27641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:33.576886 27641 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:39807
I20260811 14:30:33.576922 27717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:39807 every 8 connection(s)
I20260811 14:30:33.577574 27641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:33.578315 27718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:33.580155 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27641
I20260811 14:30:33.580245 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:33.580448 24299 external_mini_cluster.cc:1428] Setting key bd27d225bcf89aee03e102299f9ac779
I20260811 14:30:33.581182 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0: Bootstrap starting.
I20260811 14:30:33.581995 27718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:33.582369 27718 log.cc:824] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:33.583117 27718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0: No bootstrap required, opened a new log
I20260811 14:30:33.585099 27718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7506bcba91074765a06d585349e23ef0" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39807 } }
I20260811 14:30:33.585281 27718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:33.585346 27718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7506bcba91074765a06d585349e23ef0, State: Initialized, Role: FOLLOWER
I20260811 14:30:33.585500 27718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7506bcba91074765a06d585349e23ef0" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39807 } }
I20260811 14:30:33.585601 27718 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:33.585640 27718 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:33.585690 27718 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:33.586385 27718 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7506bcba91074765a06d585349e23ef0" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39807 } }
I20260811 14:30:33.586506 27718 leader_election.cc:302] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7506bcba91074765a06d585349e23ef0; no voters:
I20260811 14:30:33.586721 27718 leader_election.cc:288] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:33.587015 27718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 14:30:33 dist-test-slave-cbsf krb5kdc[27625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:33.588903 27724 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 14:30:33.589058 27736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:30:33.591300 27736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:30:33.591377 27724 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [term 1 LEADER]: Becoming Leader. State: Replica: 7506bcba91074765a06d585349e23ef0, State: Running, Role: LEADER
I20260811 14:30:33.591516 27724 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7506bcba91074765a06d585349e23ef0" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39807 } }
I20260811 14:30:33.591885 27724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7506bcba91074765a06d585349e23ef0. Latest consensus state: current_term: 1 leader_uuid: "7506bcba91074765a06d585349e23ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7506bcba91074765a06d585349e23ef0" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39807 } } }
I20260811 14:30:33.591995 27724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:33.592250 27740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:33.592458 27740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:33.593173 27723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7506bcba91074765a06d585349e23ef0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7506bcba91074765a06d585349e23ef0" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39807 } } }
I20260811 14:30:33.593276 27723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:33.594062 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:33.581070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46988 (local address 127.23.186.254:39807)
0811 14:30:33.581302 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:33.581313 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:33.581349 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:30:33.581994 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:33.582005 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:33.582075 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:33.582249 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:33.582919 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:33.583917 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:33.584726 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:33.585177 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:33.585313 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:33.588315 (+ 3002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:33.588321 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:33.588321 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:33.588335 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:33.591790 (+ 3455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:33.592387 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:33.592390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:33.592391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:33.592500 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:33.592862 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:33.592863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:33.592864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:33.593224 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:33.593384 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:33.593673 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 14:30:33.593793 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
I20260811 14:30:33.594892 27740 catalog_manager.cc:1347] Generated new cluster ID: 7c5aa390bd0e41799336467a5256ca79
I20260811 14:30:33.594949 27740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:33.595065 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:30:33.605077 27740 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:33.605893 27740 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:33.611568 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:33.607779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.23.186.254:39807)
0811 14:30:33.607830 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:33.607833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:33.607971 (+ 138us) server_negotiation.cc:407] Connection header received
0811 14:30:33.608069 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:33.608070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:33.608081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:33.608132 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:33.608554 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:33.609155 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:33.609958 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:33.610196 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:33.610278 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:33.611065 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:33.611066 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:33.611066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:33.611281 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:33.611299 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:33.611305 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:33.611376 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:33.611442 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 14:30:33.611472 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:30:33.618608 27740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7506bcba91074765a06d585349e23ef0: Generated new TSK 0
I20260811 14:30:33.618881 27740 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:30:33.624902 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27641
2026-08-11T14:30:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1453 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:33 dist-test-slave-cbsf krb5kdc[27755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:33 dist-test-slave-cbsf krb5kdc[27755](info): set up 2 sockets
Aug 11 14:30:33 dist-test-slave-cbsf krb5kdc[27755](info): commencing operation
krb5kdc: starting...
W20260811 14:30:34.792599 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:34 dist-test-slave-cbsf krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:34.928368 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:43191
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:43191
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:35.019922 27769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:35.020224 27769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:35.020270 27769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:35.021682 27769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:35.021775 27769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:35.021800 27769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:35.021836 27769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:35.021864 27769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:35.023597 27769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:43191
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:43191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:35.023934 27769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:35.024209 27769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:35.029533 27779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.029548 27777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.029554 27775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:35.029709 27769 server_base.cc:1034] running on GCE node
I20260811 14:30:35.030018 27769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:35.030367 27769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:35.031571 27769 hybrid_clock.cc:648] HybridClock initialized: now 1786458635031539 us; error 53 us; skew 500 ppm
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:35.034711 27769 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:35.035264 27769 webserver.cc:460] Webserver started at http://127.23.186.254:39315/ using document root <none> and password file <none>
I20260811 14:30:35.035514 27769 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:35.035564 27769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:35.035681 27769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:35.036996 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "c767184fe98087b529b2a7d28b6a8c41"
server_key_iv: "a9ecb3f6853521519352649fbe502b2a"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.037380 27769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "c767184fe98087b529b2a7d28b6a8c41"
server_key_iv: "a9ecb3f6853521519352649fbe502b2a"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.038908 27769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:35.039705 27787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.039903 27769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:35.039997 27769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "c767184fe98087b529b2a7d28b6a8c41"
server_key_iv: "a9ecb3f6853521519352649fbe502b2a"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.040066 27769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:35.069285 27769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:35.071835 27769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:35.071969 27769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:35.076145 27769 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:43191
I20260811 14:30:35.076215 27851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:43191 every 8 connection(s)
I20260811 14:30:35.076588 27769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:35.077805 27852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:35.080274 27852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2: Bootstrap starting.
I20260811 14:30:35.080930 27852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:35.081238 27852 log.cc:824] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:35.082024 27852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2: No bootstrap required, opened a new log
I20260811 14:30:35.083648 27852 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43191 } }
I20260811 14:30:35.083799 27852 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:35.083848 27852 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae3a15ecdf3a45e2909ed85e15d8ddd2, State: Initialized, Role: FOLLOWER
I20260811 14:30:35.084013 27852 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43191 } }
I20260811 14:30:35.084080 27852 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:35.084121 27852 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:35.084177 27852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:35.084944 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27769
I20260811 14:30:35.085034 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:35.085233 24299 external_mini_cluster.cc:1428] Setting key ed4d3265c3aaad9f03988df8a140a66b
I20260811 14:30:35.085246 27852 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43191 } }
I20260811 14:30:35.085390 27852 leader_election.cc:302] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae3a15ecdf3a45e2909ed85e15d8ddd2; no voters:
I20260811 14:30:35.085606 27852 leader_election.cc:288] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:35.085640 27855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:35.085870 27855 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [term 1 LEADER]: Becoming Leader. State: Replica: ae3a15ecdf3a45e2909ed85e15d8ddd2, State: Running, Role: LEADER
I20260811 14:30:35.085963 27852 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:35.086009 27855 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43191 } }
I20260811 14:30:35.086419 27858 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae3a15ecdf3a45e2909ed85e15d8ddd2. Latest consensus state: current_term: 1 leader_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43191 } } }
I20260811 14:30:35.086424 27857 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae3a15ecdf3a45e2909ed85e15d8ddd2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43191 } } }
I20260811 14:30:35.086556 27858 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:35.086571 27857 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2 [sys.catalog]: This master's current role is: LEADER
W20260811 14:30:35.088083 27876 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:30:35.088145 27876 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:30:35.088248 27867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:35.088464 27867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:35.090255 27867 catalog_manager.cc:1347] Generated new cluster ID: 29a860d896844ff6a91f38651940ae49
I20260811 14:30:35.090315 27867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458634, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:35.098779 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.085762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.23.186.254:43191)
0811 14:30:35.085973 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:35.085983 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:35.086018 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:30:35.086147 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:35.086154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:35.086214 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:35.087055 (+ 841us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:35.087164 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.088661 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.089726 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.090360 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.090500 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.095123 (+ 4623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:35.095128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:35.095128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:35.095143 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:35.097048 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.097491 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.097495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.097496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.097621 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.097847 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.097848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.097849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.098125 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:35.098318 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:35.098388 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 14:30:35.098566 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260811 14:30:35.107427 27867 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:35.108021 27867 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:35.111094 27867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae3a15ecdf3a45e2909ed85e15d8ddd2: Generated new TSK 0
I20260811 14:30:35.111326 27867 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:35.151278 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43191
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:35.260982 27882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:35.261274 27882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:35.261322 27882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:35.262665 27882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:35.262753 27882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:35.262804 27882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:35.264544 27882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:35.264894 27882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:35.265168 27882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:35.270851 27891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:35.270965 27882 server_base.cc:1034] running on GCE node
W20260811 14:30:35.271076 27888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.270853 27889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:35.271340 27882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:35.271643 27882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:35.272819 27882 hybrid_clock.cc:648] HybridClock initialized: now 1786458635272808 us; error 48 us; skew 500 ppm
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:35.275626 27882 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:35.276149 27882 webserver.cc:460] Webserver started at http://127.23.186.193:37913/ using document root <none> and password file <none>
I20260811 14:30:35.276425 27882 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:35.276501 27882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:35.276664 27882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:35.277830 27882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "0443b9cff3bf40a78f2d303611e464da"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "0cd8781fc73cda6405b1bd7d8b691db2"
server_key_iv: "884506487b7b7b12894bf19c57f58356"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.278221 27882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "0443b9cff3bf40a78f2d303611e464da"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "0cd8781fc73cda6405b1bd7d8b691db2"
server_key_iv: "884506487b7b7b12894bf19c57f58356"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.279675 27882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:35.280521 27899 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.280755 27882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:35.280874 27882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "0443b9cff3bf40a78f2d303611e464da"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "0cd8781fc73cda6405b1bd7d8b691db2"
server_key_iv: "884506487b7b7b12894bf19c57f58356"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.280959 27882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:35.298166 27882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:35.300911 27882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:35.301113 27882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:35.301378 27882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:35.301725 27882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:35.301782 27882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.301829 27882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:35.301854 27882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.308063 27882 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:39449
I20260811 14:30:35.308108 28036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:39449 every 8 connection(s)
I20260811 14:30:35.308463 27882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:35.308856 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 27882
I20260811 14:30:35.308970 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:35.309154 24299 external_mini_cluster.cc:1428] Setting key 26f25235ed16f04e2f9b9757a1433798
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
I20260811 14:30:35.320433 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.309345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:51753 (local address 127.23.186.254:43191)
0811 14:30:35.309402 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:35.309405 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:35.309698 (+ 293us) server_negotiation.cc:407] Connection header received
0811 14:30:35.310402 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:35.310403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:35.310417 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:35.310469 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:35.311925 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.312795 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.313906 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.314256 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.314348 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.317190 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:35.317191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:35.317192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:35.317201 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:35.318754 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.319336 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.319337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.319338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.319394 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.319706 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.319706 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.319707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.319826 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:35.319878 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:35.320229 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 14:30:35.320354 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:30:35.320557 28042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.309337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:43191 (local address 127.23.186.193:51753)
0811 14:30:35.309555 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:35.309613 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:35.310297 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:35.310561 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:35.310566 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:35.311471 (+ 905us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:35.311782 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:35.311788 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.312928 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.312928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:35.313790 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:35.313793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.314360 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.314361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:35.314479 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.315147 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:35.315156 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:35.317041 (+ 1885us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:35.318857 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:35.318867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:35.318871 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:35.319212 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:35.319473 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:35.319474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:35.319474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:35.319642 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:35.319887 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:35.319892 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:35.320025 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 14:30:35.320185 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 14:30:35.320347 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260811 14:30:35.321151 28037 heartbeater.cc:347] Connected to a master server at 127.23.186.254:43191
I20260811 14:30:35.321269 28037 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:35.321518 28037 heartbeater.cc:511] Master 127.23.186.254:43191 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
I20260811 14:30:35.322150 27810 ts_manager.cc:194] Registered new tserver with Master: 0443b9cff3bf40a78f2d303611e464da (127.23.186.193:39449)
I20260811 14:30:35.322954 27810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:51753
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:35.337829 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43191
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:35.422930 28045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:35.423172 28045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:35.423198 28045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:35.424489 28045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:35.424544 28045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:35.424572 28045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:35.426198 28045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:35.426406 28045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:35.426589 28045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:35.431931 28050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.431932 28051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.431905 28053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:35.432173 28045 server_base.cc:1034] running on GCE node
I20260811 14:30:35.432369 28045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:35.432596 28045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:35.433756 28045 hybrid_clock.cc:648] HybridClock initialized: now 1786458635433734 us; error 30 us; skew 500 ppm
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:35.436463 28045 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:35.437058 28045 webserver.cc:460] Webserver started at http://127.23.186.194:39935/ using document root <none> and password file <none>
I20260811 14:30:35.437321 28045 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:35.437393 28045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:35.437548 28045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:35.438671 28045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "32a10fe2771342b99934e02a595a10fd"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "0fea34d36d098aafe242f56ccf51cf95"
server_key_iv: "a73369436292a760e91f48e052cbc898"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.439055 28045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "32a10fe2771342b99934e02a595a10fd"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "0fea34d36d098aafe242f56ccf51cf95"
server_key_iv: "a73369436292a760e91f48e052cbc898"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.440495 28045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:35.441370 28061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.441546 28045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:35.441668 28045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "32a10fe2771342b99934e02a595a10fd"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "0fea34d36d098aafe242f56ccf51cf95"
server_key_iv: "a73369436292a760e91f48e052cbc898"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.441751 28045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:35.452020 28045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:35.454217 28045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:35.454407 28045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:35.454700 28045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:35.455047 28045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:35.455106 28045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.455153 28045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:35.455178 28045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.461627 28045 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:45701
I20260811 14:30:35.461675 28205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:45701 every 8 connection(s)
I20260811 14:30:35.462037 28045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:35.464277 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 28045
I20260811 14:30:35.464423 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:35.464666 24299 external_mini_cluster.cc:1428] Setting key 25c01ef94723a085c868df46e57be5bf
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:35.473909 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.462882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:57551 (local address 127.23.186.254:43191)
0811 14:30:35.462971 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:35.462973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:35.463256 (+ 283us) server_negotiation.cc:407] Connection header received
0811 14:30:35.463796 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:35.463797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:35.463809 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:35.463864 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:35.465397 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.465945 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.467631 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.467959 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.468041 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.470654 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:35.470655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:35.470656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:35.470663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:35.472170 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.472790 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.472790 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.472791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.472848 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.473182 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.473182 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.473183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.473297 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:35.473346 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:35.473765 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 14:30:35.473839 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:30:35.474278 28209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.462853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:43191 (local address 127.23.186.194:57551)
0811 14:30:35.463095 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:35.463145 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:35.463695 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:35.463953 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:35.463957 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:35.464998 (+ 1041us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:35.465267 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:35.465274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.466055 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.466056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:35.467515 (+ 1459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:35.467519 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.468062 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.468064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:35.468186 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.468875 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:35.468887 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:35.470531 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:35.472275 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:35.472286 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:35.472290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:35.472660 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:35.472925 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:35.472927 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:35.472927 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:35.473100 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:35.473370 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:35.473378 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:35.473589 (+ 211us) client_negotiation.cc:769] Sending connection context
0811 14:30:35.473720 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 14:30:35.473958 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":122,"threads_started":1}
I20260811 14:30:35.474907 28207 heartbeater.cc:347] Connected to a master server at 127.23.186.254:43191
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
I20260811 14:30:35.475041 28207 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:35.475289 28207 heartbeater.cc:511] Master 127.23.186.254:43191 requested a full tablet report, sending...
I20260811 14:30:35.475742 27810 ts_manager.cc:194] Registered new tserver with Master: 32a10fe2771342b99934e02a595a10fd (127.23.186.194:45701)
I20260811 14:30:35.476176 27810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:57551
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:35.495764 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43191
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:35.584372 28212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:35.584635 28212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:35.584663 28212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:35.586069 28212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:35.586120 28212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:35.586150 28212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:35.588119 28212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43191
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:35.588407 28212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:35.588680 28212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:35.594002 28218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.594081 28224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:35.594008 28220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:35.594223 28212 server_base.cc:1034] running on GCE node
I20260811 14:30:35.594645 28212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:35.594920 28212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:35.596117 28212 hybrid_clock.cc:648] HybridClock initialized: now 1786458635596101 us; error 50 us; skew 500 ppm
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:35.599045 28212 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:35.599596 28212 webserver.cc:460] Webserver started at http://127.23.186.195:45399/ using document root <none> and password file <none>
I20260811 14:30:35.599859 28212 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:35.599908 28212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:35.600092 28212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:35.601295 28212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "89c6af1d25694c06ad42e5be34754fea"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "a13996eea0095f9eda6fd459d773155c"
server_key_iv: "b78b7b1bb57d1ed1dcfe9e0b77e93816"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.601677 28212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "89c6af1d25694c06ad42e5be34754fea"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "a13996eea0095f9eda6fd459d773155c"
server_key_iv: "b78b7b1bb57d1ed1dcfe9e0b77e93816"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.603085 28212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:35.603881 28231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.604077 28212 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:35.604192 28212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "89c6af1d25694c06ad42e5be34754fea"
format_stamp: "Formatted at 2026-08-11 14:30:35 on dist-test-slave-cbsf"
server_key: "a13996eea0095f9eda6fd459d773155c"
server_key_iv: "b78b7b1bb57d1ed1dcfe9e0b77e93816"
server_key_version: "encryptionkey@0"
I20260811 14:30:35.604274 28212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:35.619750 28212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:35.622027 28212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:35.622222 28212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:35.622506 28212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:35.622881 28212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:35.622939 28212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.622987 28212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:35.623009 28212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:35.629734 28212 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:39979
I20260811 14:30:35.629776 28364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:39979 every 8 connection(s)
I20260811 14:30:35.630146 28212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:35.632901 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 28212
I20260811 14:30:35.633009 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:35.633193 24299 external_mini_cluster.cc:1428] Setting key 8b13bcc48a2375b4f045fe73fd593f76
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:35.640473 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.630998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:46891 (local address 127.23.186.254:43191)
0811 14:30:35.631088 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:35.631091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:35.631259 (+ 168us) server_negotiation.cc:407] Connection header received
0811 14:30:35.631769 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:35.631771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:35.631783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:35.631840 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:35.632968 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.633718 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.634913 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.635110 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.635182 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.637664 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:35.637666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:35.637666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:35.637673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:35.639191 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.639607 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.639607 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.639608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.639663 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.639895 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.639896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.639896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.640009 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:35.640061 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:35.640329 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 14:30:35.640403 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 14:30:35.640702 28367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.630933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:43191 (local address 127.23.186.195:46891)
0811 14:30:35.631130 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:35.631171 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:35.631668 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:35.631924 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:35.631928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:35.632558 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:35.632866 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:35.632872 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.633826 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.633828 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:35.634809 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:35.634812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.635223 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.635225 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:35.635358 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.635884 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:35.635893 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:35.637562 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:35.639279 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:35.639288 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:35.639290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:35.639533 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:35.639724 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:35.639725 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:35.639725 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:35.639837 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:35.640065 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:35.640070 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:35.640212 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 14:30:35.640300 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:30:35.640434 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":110,"threads_started":1}
I20260811 14:30:35.641323 28365 heartbeater.cc:347] Connected to a master server at 127.23.186.254:43191
I20260811 14:30:35.641430 28365 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:35.641595 28365 heartbeater.cc:511] Master 127.23.186.254:43191 requested a full tablet report, sending...
I20260811 14:30:35.641964 27810 ts_manager.cc:194] Registered new tserver with Master: 89c6af1d25694c06ad42e5be34754fea (127.23.186.195:39979)
I20260811 14:30:35.642373 27810 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:46891
I20260811 14:30:35.645064 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[27755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458635, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:35.671054 27859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:35.663156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57518 (local address 127.23.186.254:43191)
0811 14:30:35.663246 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:35.663249 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:35.663390 (+ 141us) server_negotiation.cc:407] Connection header received
0811 14:30:35.663425 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:35.663426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:35.663438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:35.663486 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:35.664093 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.664587 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.665414 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:35.665613 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:35.665687 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:35.668123 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:35.668125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:35.668125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:35.668132 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:35.669653 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.670031 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.670033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.670033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.670107 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:35.670395 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:35.670396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:35.670396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:35.670582 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:35.670699 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:35.670834 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 14:30:35.670927 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 14:30:35.671343 27810 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:57518
I20260811 14:30:35.671811 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27882
I20260811 14:30:35.682924 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 28045
I20260811 14:30:35.698346 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 28212
I20260811 14:30:35.712440 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 27769
2026-08-11T14:30:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2085 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[28377](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[28377](info): set up 2 sockets
Aug 11 14:30:35 dist-test-slave-cbsf krb5kdc[28377](info): commencing operation
krb5kdc: starting...
W20260811 14:30:36.840813 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:36 dist-test-slave-cbsf krb5kdc[28377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:36.979807 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37275
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:37275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:37.070947 28390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:37.071272 28390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:37.071313 28390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:37.072643 28390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:37.072746 28390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:37.072774 28390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:37.072803 28390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:37.072830 28390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:37.074506 28390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:37275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:37.074826 28390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:37.075088 28390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:37.080642 28396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.080622 28398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.080622 28401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:37.080915 28390 server_base.cc:1034] running on GCE node
I20260811 14:30:37.081164 28390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:37.081437 28390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:37.082623 28390 hybrid_clock.cc:648] HybridClock initialized: now 1786458637082613 us; error 40 us; skew 500 ppm
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:37.091130 28390 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:37.091763 28390 webserver.cc:460] Webserver started at http://127.23.186.254:40927/ using document root <none> and password file <none>
I20260811 14:30:37.092022 28390 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:37.092099 28390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:37.092262 28390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:37.093638 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "6682268f52f642c896da95cc4f1a7bd8"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "8af422f022f758e149db53c6432ef2b7"
server_key_iv: "8c62b1eb3836f1f42502632b5c7e796c"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.094060 28390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "6682268f52f642c896da95cc4f1a7bd8"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "8af422f022f758e149db53c6432ef2b7"
server_key_iv: "8c62b1eb3836f1f42502632b5c7e796c"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.095605 28390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:37.096493 28409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.096841 28390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:37.096946 28390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "6682268f52f642c896da95cc4f1a7bd8"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "8af422f022f758e149db53c6432ef2b7"
server_key_iv: "8c62b1eb3836f1f42502632b5c7e796c"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.097018 28390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:37.115690 28390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:37.118126 28390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:37.118268 28390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:37.122946 28390 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:37275
I20260811 14:30:37.123028 28469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:37275 every 8 connection(s)
I20260811 14:30:37.123601 28390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:37.124419 28470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:37.126935 28470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8: Bootstrap starting.
I20260811 14:30:37.127259 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 28390
I20260811 14:30:37.127377 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:37.127583 24299 external_mini_cluster.cc:1428] Setting key a0de08da08dd72cb63f179ec6904d89d
I20260811 14:30:37.127717 28470 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:37.128088 28470 log.cc:824] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:37.129045 28470 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8: No bootstrap required, opened a new log
I20260811 14:30:37.131520 28470 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6682268f52f642c896da95cc4f1a7bd8" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37275 } }
I20260811 14:30:37.131707 28470 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:37.131757 28470 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6682268f52f642c896da95cc4f1a7bd8, State: Initialized, Role: FOLLOWER
I20260811 14:30:37.131906 28470 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6682268f52f642c896da95cc4f1a7bd8" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37275 } }
I20260811 14:30:37.131999 28470 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:37.132050 28470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:37.132102 28470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:37.132797 28470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6682268f52f642c896da95cc4f1a7bd8" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37275 } }
I20260811 14:30:37.132910 28470 leader_election.cc:302] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6682268f52f642c896da95cc4f1a7bd8; no voters:
I20260811 14:30:37.133087 28470 leader_election.cc:288] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:37.133172 28476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:37.133363 28470 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:37.133368 28476 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [term 1 LEADER]: Becoming Leader. State: Replica: 6682268f52f642c896da95cc4f1a7bd8, State: Running, Role: LEADER
I20260811 14:30:37.133499 28476 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6682268f52f642c896da95cc4f1a7bd8" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37275 } }
I20260811 14:30:37.133862 28479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6682268f52f642c896da95cc4f1a7bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6682268f52f642c896da95cc4f1a7bd8" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37275 } } }
I20260811 14:30:37.133842 28476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6682268f52f642c896da95cc4f1a7bd8. Latest consensus state: current_term: 1 leader_uuid: "6682268f52f642c896da95cc4f1a7bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6682268f52f642c896da95cc4f1a7bd8" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37275 } } }
I20260811 14:30:37.134038 28479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:37.134050 28476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:37.134860 28488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:37.135412 28488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:37.137444 28488 catalog_manager.cc:1347] Generated new cluster ID: b6af55b759d848fdb5d59f2b6c3662a6
I20260811 14:30:37.137504 28488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:37.140323 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.128229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.23.186.254:37275)
0811 14:30:37.128492 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:37.128500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:37.128526 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:30:37.129206 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:37.129217 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:37.129282 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:37.129430 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:37.130112 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.131606 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.132627 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.133341 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.135132 (+ 1791us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.136716 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:37.136721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:37.136721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:37.136735 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:37.138676 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.139086 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.139089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.139090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.139185 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.139398 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.139399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.139399 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.139662 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:37.139841 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:37.139943 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 14:30:37.140101 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":140,"threads_started":1}
I20260811 14:30:37.146581 28488 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:37.147358 28488 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:37.153961 28488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6682268f52f642c896da95cc4f1a7bd8: Generated new TSK 0
I20260811 14:30:37.154256 28488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:37.185762 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37275
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:37.274637 28501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:37.274880 28501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:37.274909 28501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:37.276201 28501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:37.276260 28501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:37.276290 28501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:37.278162 28501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:37.278489 28501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:37.278753 28501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:30:37.284458 28501 server_base.cc:1034] running on GCE node
W20260811 14:30:37.284425 28507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.284448 28509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.284425 28506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:37.284893 28501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:37.285141 28501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:37.286296 28501 hybrid_clock.cc:648] HybridClock initialized: now 1786458637286289 us; error 33 us; skew 500 ppm
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:37.289273 28501 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:37.289740 28501 webserver.cc:460] Webserver started at http://127.23.186.193:42497/ using document root <none> and password file <none>
I20260811 14:30:37.289958 28501 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:37.290004 28501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:37.290098 28501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:37.291200 28501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "8f677b40fa754dd8aa5fc0328a873fd4"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "2e41624736a40097b237e1ace3c4e26b"
server_key_iv: "f6b1f795f8daf18c67f595893f244a75"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.291522 28501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "8f677b40fa754dd8aa5fc0328a873fd4"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "2e41624736a40097b237e1ace3c4e26b"
server_key_iv: "f6b1f795f8daf18c67f595893f244a75"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.293084 28501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260811 14:30:37.293883 28517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.294088 28501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:37.294224 28501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "8f677b40fa754dd8aa5fc0328a873fd4"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "2e41624736a40097b237e1ace3c4e26b"
server_key_iv: "f6b1f795f8daf18c67f595893f244a75"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.294312 28501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:37.313519 28501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:37.316150 28501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:37.316334 28501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:37.316591 28501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:37.316990 28501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:37.317055 28501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.317116 28501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:37.317162 28501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.323376 28501 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:41633
I20260811 14:30:37.323426 28651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:41633 every 8 connection(s)
I20260811 14:30:37.323786 28501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:37.332609 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 28501
I20260811 14:30:37.332746 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:37.332971 24299 external_mini_cluster.cc:1428] Setting key 046b486d1c8e2abd981dcb86c9eec841
I20260811 14:30:37.334686 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.324647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:53335 (local address 127.23.186.254:37275)
0811 14:30:37.324742 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:37.324744 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:37.324985 (+ 241us) server_negotiation.cc:407] Connection header received
0811 14:30:37.325960 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:37.325961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:37.325971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:37.326035 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:37.326709 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.327244 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.328339 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.328556 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.328630 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.331093 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:37.331095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:37.331095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:37.331107 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:37.332861 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.333472 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.333473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.333473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.333532 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.333842 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.333842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.333843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.333984 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:37.334054 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:37.334502 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 14:30:37.334617 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 14:30:37.334934 28654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.324614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37275 (local address 127.23.186.193:53335)
0811 14:30:37.324838 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:37.324887 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:37.325859 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:37.326098 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:37.326102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:37.326399 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:37.326623 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:37.326628 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.327337 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.327338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:37.328213 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:37.328215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.328647 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.328648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:37.328797 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.329363 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:37.329373 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:37.330966 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:37.332956 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:37.332966 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:37.332968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:37.333396 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:37.333601 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:37.333602 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:37.333602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:37.333773 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:37.334066 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:37.334073 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:37.334271 (+ 198us) client_negotiation.cc:769] Sending connection context
0811 14:30:37.334398 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 14:30:37.334636 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":127,"threads_started":1}
I20260811 14:30:37.335454 28652 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37275
I20260811 14:30:37.335609 28652 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:37.335824 28652 heartbeater.cc:511] Master 127.23.186.254:37275 requested a full tablet report, sending...
I20260811 14:30:37.336711 28430 ts_manager.cc:194] Registered new tserver with Master: 8f677b40fa754dd8aa5fc0328a873fd4 (127.23.186.193:41633)
I20260811 14:30:37.337539 28430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:53335
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:37.361929 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37275
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:37.451876 28658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:37.452178 28658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:37.452222 28658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:37.453557 28658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:37.453660 28658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:37.453711 28658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:37.455442 28658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:37.455788 28658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:37.456055 28658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:37.461979 28669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.461988 28663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:37.462219 28658 server_base.cc:1034] running on GCE node
W20260811 14:30:37.461978 28665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:37.462529 28658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:37.462821 28658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:37.463996 28658 hybrid_clock.cc:648] HybridClock initialized: now 1786458637463998 us; error 44 us; skew 500 ppm
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:37.466715 28658 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:37.467260 28658 webserver.cc:460] Webserver started at http://127.23.186.194:40775/ using document root <none> and password file <none>
I20260811 14:30:37.467520 28658 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:37.467597 28658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:37.467782 28658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:37.469080 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "b402a8ef878b45f88014291a3dad3119"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "78a26858705e89ed8dccafc8ad8df761"
server_key_iv: "ed5c355e0558cfbea2232e555ab6a2be"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.469477 28658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "b402a8ef878b45f88014291a3dad3119"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "78a26858705e89ed8dccafc8ad8df761"
server_key_iv: "ed5c355e0558cfbea2232e555ab6a2be"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.470999 28658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:37.471840 28677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.472020 28658 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:37.472148 28658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "b402a8ef878b45f88014291a3dad3119"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "78a26858705e89ed8dccafc8ad8df761"
server_key_iv: "ed5c355e0558cfbea2232e555ab6a2be"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.472251 28658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:37.481199 28658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:37.483243 28658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:37.483446 28658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:37.483708 28658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:37.484058 28658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:37.484131 28658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.484179 28658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:37.484202 28658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.490443 28658 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:43439
I20260811 14:30:37.490485 28812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:43439 every 8 connection(s)
I20260811 14:30:37.490844 28658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:37.498713 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 28658
I20260811 14:30:37.498818 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:37.499008 24299 external_mini_cluster.cc:1428] Setting key 528842725a74a3c7a7e685e287a7dd4b
I20260811 14:30:37.502355 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.491602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:44485 (local address 127.23.186.254:37275)
0811 14:30:37.491662 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:37.491665 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:37.491867 (+ 202us) server_negotiation.cc:407] Connection header received
0811 14:30:37.492930 (+ 1063us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:37.492932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:37.492945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:37.493016 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:37.493711 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.494372 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.495534 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.495746 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.495821 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.498384 (+ 2563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:37.498387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:37.498387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:37.498400 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:37.500776 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.501276 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.501277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.501278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.501348 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.501619 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.501620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.501621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.501762 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:37.501842 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:37.502163 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 14:30:37.502254 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:30:37.502581 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.491537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37275 (local address 127.23.186.194:44485)
0811 14:30:37.491705 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:37.491750 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:37.492804 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:37.493097 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:37.493102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:37.493380 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:37.493600 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:37.493604 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.494541 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.494542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:37.495395 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:37.495398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.495840 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.495841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:37.495953 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.496496 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:37.496505 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:37.498223 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:37.500881 (+ 2658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:37.500892 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:37.500895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:37.501179 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:37.501414 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:37.501415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:37.501415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:37.501541 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:37.501830 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:37.501838 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:37.502002 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 14:30:37.502133 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 14:30:37.502318 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":95,"threads_started":1}
I20260811 14:30:37.503250 28813 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37275
I20260811 14:30:37.503360 28813 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:37.503582 28813 heartbeater.cc:511] Master 127.23.186.254:37275 requested a full tablet report, sending...
I20260811 14:30:37.504061 28430 ts_manager.cc:194] Registered new tserver with Master: b402a8ef878b45f88014291a3dad3119 (127.23.186.194:43439)
I20260811 14:30:37.504659 28430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:44485
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:37.527609 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37275
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:45399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:37.615607 28818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:37.615852 28818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:37.615881 28818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:37.617247 28818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:37.617303 28818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:37.617334 28818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:37.619067 28818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:37.619383 28818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:37.619663 28818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:37.625173 28825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.625164 28828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:37.625165 28824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:37.625460 28818 server_base.cc:1034] running on GCE node
I20260811 14:30:37.625836 28818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:37.626094 28818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:37.627261 28818 hybrid_clock.cc:648] HybridClock initialized: now 1786458637627254 us; error 36 us; skew 500 ppm
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:37.630098 28818 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:37.630642 28818 webserver.cc:460] Webserver started at http://127.23.186.195:39399/ using document root <none> and password file <none>
I20260811 14:30:37.630884 28818 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:37.630929 28818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:37.631035 28818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:37.632122 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "5803912a6c7b43789b6606862de152b3"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "fb734ebcd4f3fab4768542876bfb6d44"
server_key_iv: "4bb6e64ec383097ad6720103f8173f53"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.632441 28818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "5803912a6c7b43789b6606862de152b3"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "fb734ebcd4f3fab4768542876bfb6d44"
server_key_iv: "4bb6e64ec383097ad6720103f8173f53"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.634150 28818 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:30:37.635059 28836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.635273 28818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:37.635380 28818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "5803912a6c7b43789b6606862de152b3"
format_stamp: "Formatted at 2026-08-11 14:30:37 on dist-test-slave-cbsf"
server_key: "fb734ebcd4f3fab4768542876bfb6d44"
server_key_iv: "4bb6e64ec383097ad6720103f8173f53"
server_key_version: "encryptionkey@0"
I20260811 14:30:37.635488 28818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:37.654309 28818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:37.656553 28818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:37.656769 28818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:37.657055 28818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:37.657421 28818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:37.657480 28818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.657522 28818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:37.657568 28818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:37.663717 28818 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:42083
I20260811 14:30:37.663774 28971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:42083 every 8 connection(s)
I20260811 14:30:37.664186 28818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:37.674355 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 28818
I20260811 14:30:37.674451 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:37.674675 24299 external_mini_cluster.cc:1428] Setting key d1596496fed9d09e5caf68ad41d1476e
I20260811 14:30:37.675220 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.665115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:60327 (local address 127.23.186.254:37275)
0811 14:30:37.665180 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:37.665182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:37.665419 (+ 237us) server_negotiation.cc:407] Connection header received
0811 14:30:37.666460 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:37.666461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:37.666475 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:37.666525 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:37.667236 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.667764 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.668933 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.669146 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.669232 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.671687 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:37.671689 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:37.671689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:37.671707 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:37.673257 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.673903 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.673905 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.673905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.673976 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.674288 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.674290 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.674290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.674460 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:37.674530 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:37.674955 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 14:30:37.675064 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:30:37.675493 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.665073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37275 (local address 127.23.186.195:60327)
0811 14:30:37.665292 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:37.665332 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:37.666329 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:37.666611 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:37.666615 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:37.666914 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:37.667126 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:37.667130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.667879 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.667880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:37.668841 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:37.668844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.669244 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.669245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:37.669390 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.669960 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:37.669970 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:37.671559 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:37.673484 (+ 1925us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:37.673514 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:37.673517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:37.673801 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:37.674063 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:37.674064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:37.674064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:37.674218 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:37.674561 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:37.674573 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:37.674753 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 14:30:37.674901 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 14:30:37.675188 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":99,"threads_started":1}
I20260811 14:30:37.676138 28972 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37275
I20260811 14:30:37.676292 28972 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:37.676491 28972 heartbeater.cc:511] Master 127.23.186.254:37275 requested a full tablet report, sending...
I20260811 14:30:37.676908 28430 ts_manager.cc:194] Registered new tserver with Master: 5803912a6c7b43789b6606862de152b3 (127.23.186.195:42083)
I20260811 14:30:37.677330 28430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:60327
I20260811 14:30:37.686965 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:37.715159 28980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.706733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.23.186.193:41633)
0811 14:30:37.706960 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:37.706966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:37.706980 (+ 14us) server_negotiation.cc:407] Connection header received
0811 14:30:37.707009 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:37.707014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:37.707030 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:37.707106 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:37.707729 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.708481 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.709314 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.709696 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.709760 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.712125 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:37.712138 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:37.712139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:37.712147 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:37.714005 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.714381 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.714384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.714384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.714444 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.714609 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.714609 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.714610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.714773 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:37.714885 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:37.714965 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:30:37.715036 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":110,"threads_started":1}
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28377](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458637, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:37.724066 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:37.716813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59866 (local address 127.23.186.254:37275)
0811 14:30:37.716878 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:37.716880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:37.716895 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:30:37.716921 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:37.716922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:37.716931 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:37.716979 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:37.717470 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.717976 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.718656 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:37.718841 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:37.718892 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:37.721400 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:37.721402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:37.721402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:37.721411 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:37.722814 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.723197 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.723198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.723199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.723266 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:37.723454 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:37.723455 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:37.723455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:37.723636 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:37.723723 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:37.723847 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 14:30:37.723928 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 14:30:37.724354 28430 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59866
I20260811 14:30:37.724840 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 28501
I20260811 14:30:37.734432 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 28658
I20260811 14:30:37.742488 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 28818
I20260811 14:30:37.750841 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 28390
2026-08-11T14:30:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2040 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28986](info): set up 2 sockets
Aug 11 14:30:37 dist-test-slave-cbsf krb5kdc[28986](info): commencing operation
krb5kdc: starting...
W20260811 14:30:38.877383 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:38 dist-test-slave-cbsf krb5kdc[28986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:39.011816 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37065
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:37065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:39.101398 29002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:39.101645 29002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:39.101673 29002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:39.102898 29002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:39.102943 29002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:39.102958 29002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:39.102972 29002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:39.102983 29002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:39.104592 29002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:37065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:39.104856 29002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:39.105072 29002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:39.110289 29007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.110332 29008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.110334 29010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:39.110560 29002 server_base.cc:1034] running on GCE node
I20260811 14:30:39.110860 29002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:39.111199 29002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:39.112354 29002 hybrid_clock.cc:648] HybridClock initialized: now 1786458639112331 us; error 38 us; skew 500 ppm
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:39.115202 29002 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:39.115578 29002 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:39.115628 29002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:39.115729 29002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:39.116936 29002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "4884c3b42cf1421c9fad01acab212944"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "6c15f7bb57f28890a3cd72e9c609b639"
server_key_iv: "6209af900dfad51dca41190fd7df250f"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.117287 29002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "4884c3b42cf1421c9fad01acab212944"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "6c15f7bb57f28890a3cd72e9c609b639"
server_key_iv: "6209af900dfad51dca41190fd7df250f"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.118749 29002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:39.119580 29016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.119809 29002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:39.119930 29002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "4884c3b42cf1421c9fad01acab212944"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "6c15f7bb57f28890a3cd72e9c609b639"
server_key_iv: "6209af900dfad51dca41190fd7df250f"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.120014 29002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:39.139063 29002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:39.142622 29002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:39.142867 29002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:39.147809 29002 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:37065
I20260811 14:30:39.147837 29076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:37065 every 8 connection(s)
I20260811 14:30:39.148192 29002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:39.148679 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29002
I20260811 14:30:39.148850 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:39.149058 24299 external_mini_cluster.cc:1428] Setting key 463fdd917dd8a2ba89e758c3ec239c13
I20260811 14:30:39.150130 29077 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:39.154050 29077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944: Bootstrap starting.
I20260811 14:30:39.154892 29077 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:39.155231 29077 log.cc:824] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:39.156193 29077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944: No bootstrap required, opened a new log
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:39.159298 29077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4884c3b42cf1421c9fad01acab212944" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37065 } }
I20260811 14:30:39.159462 29077 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:39.159487 29077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4884c3b42cf1421c9fad01acab212944, State: Initialized, Role: FOLLOWER
I20260811 14:30:39.159687 29077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4884c3b42cf1421c9fad01acab212944" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37065 } }
I20260811 14:30:39.159795 29077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:39.159832 29077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:39.159889 29077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:39.160809 29077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4884c3b42cf1421c9fad01acab212944" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37065 } }
I20260811 14:30:39.160933 29077 leader_election.cc:302] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4884c3b42cf1421c9fad01acab212944; no voters:
I20260811 14:30:39.161147 29077 leader_election.cc:288] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:39.161213 29082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:39.161432 29082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [term 1 LEADER]: Becoming Leader. State: Replica: 4884c3b42cf1421c9fad01acab212944, State: Running, Role: LEADER
I20260811 14:30:39.161443 29077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:39.161554 29082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4884c3b42cf1421c9fad01acab212944" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37065 } }
I20260811 14:30:39.161926 29083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4884c3b42cf1421c9fad01acab212944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4884c3b42cf1421c9fad01acab212944" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37065 } } }
I20260811 14:30:39.162133 29083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:39.161911 29082 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4884c3b42cf1421c9fad01acab212944. Latest consensus state: current_term: 1 leader_uuid: "4884c3b42cf1421c9fad01acab212944" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4884c3b42cf1421c9fad01acab212944" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37065 } } }
I20260811 14:30:39.162483 29082 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:39.162667 29094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:39.163333 29094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:39.163695 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.149783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.23.186.254:37065)
0811 14:30:39.150006 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.150016 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.150052 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:30:39.150586 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.150594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.150649 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.150802 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:39.151795 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.153270 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.154504 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.155299 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.155604 (+ 305us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.158933 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:39.158941 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:39.158941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:39.158961 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:39.161222 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.161699 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.161703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.161704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.161808 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.162029 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.162030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.162030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.162434 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:39.162590 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.163225 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.163381 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260811 14:30:39.165117 29094 catalog_manager.cc:1347] Generated new cluster ID: 5ff0364aada5469698516605ecc4ad35
I20260811 14:30:39.165185 29094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:39.195060 29094 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:39.195938 29094 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:39.203970 29094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944: Generated new TSK 0
I20260811 14:30:39.204288 29094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:39.239843 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37065
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:39.346835 29106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:39.347164 29106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:39.347218 29106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:39.348889 29106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:39.348976 29106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:39.349035 29106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:39.351020 29106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:39.351388 29106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:39.351694 29106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:39.357828 29115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.357914 29112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.357792 29113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:39.358518 29106 server_base.cc:1034] running on GCE node
I20260811 14:30:39.358806 29106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:39.359131 29106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:39.360379 29106 hybrid_clock.cc:648] HybridClock initialized: now 1786458639360360 us; error 42 us; skew 500 ppm
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:39.364332 29106 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:39.364902 29106 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:39.364965 29106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:39.365121 29106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:39.366386 29106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "f515d9d2baf649278bc3cdacbd7b49e4"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "bf3a4c81c3b4930816ab202b63f80c88"
server_key_iv: "d5aaa8f7ba06448b35f2aea8a40645e6"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.366817 29106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "f515d9d2baf649278bc3cdacbd7b49e4"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "bf3a4c81c3b4930816ab202b63f80c88"
server_key_iv: "d5aaa8f7ba06448b35f2aea8a40645e6"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.368652 29106 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:30:39.370237 29123 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.370657 29106 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:39.370782 29106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "f515d9d2baf649278bc3cdacbd7b49e4"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "bf3a4c81c3b4930816ab202b63f80c88"
server_key_iv: "d5aaa8f7ba06448b35f2aea8a40645e6"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.370920 29106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:39.386408 29106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:39.389348 29106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:39.389533 29106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:39.389859 29106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:39.390269 29106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:39.390338 29106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.390391 29106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:39.390439 29106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.398981 29106 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:35895
I20260811 14:30:39.399096 29257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:35895 every 8 connection(s)
I20260811 14:30:39.399471 29106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:39.409439 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29106
I20260811 14:30:39.409621 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:39.409994 24299 external_mini_cluster.cc:1428] Setting key 951066abe99eb9223c810a0149d226a2
I20260811 14:30:39.413472 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.400534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:54041 (local address 127.23.186.254:37065)
0811 14:30:39.400728 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.400732 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.400929 (+ 197us) server_negotiation.cc:407] Connection header received
0811 14:30:39.401462 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.401464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.401478 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.401569 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:39.402776 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.403391 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.404582 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.404810 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.404872 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.407820 (+ 2948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:39.407823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:39.407823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:39.407837 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:39.410732 (+ 2895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.411644 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.411646 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.411647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.411743 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.412285 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.412286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.412286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.412490 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:39.412602 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.413150 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.413295 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 14:30:39.413784 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.400491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37065 (local address 127.23.186.193:54041)
0811 14:30:39.400731 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.400786 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.401365 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.401643 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.401647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.401962 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:39.402657 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.402664 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.403498 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.403499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.404487 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.404489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.404903 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.404904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.405031 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.405600 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:39.405610 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:39.407651 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:39.410896 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:39.410917 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:39.410921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:39.411541 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:39.411873 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:39.411875 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:39.411875 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:39.412099 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:39.412611 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:39.412628 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:39.412954 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.413093 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.413402 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
I20260811 14:30:39.414613 29258 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37065
I20260811 14:30:39.414806 29258 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:39.415060 29258 heartbeater.cc:511] Master 127.23.186.254:37065 requested a full tablet report, sending...
I20260811 14:30:39.415912 29035 ts_manager.cc:194] Registered new tserver with Master: f515d9d2baf649278bc3cdacbd7b49e4 (127.23.186.193:35895)
I20260811 14:30:39.416683 29035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:54041
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:39.445725 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37065
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:39.538619 29263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:39.538877 29263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:39.538908 29263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:39.540117 29263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:39.540172 29263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:39.540201 29263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:39.541922 29263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:39.542178 29263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:39.542392 29263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:39.548187 29270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.548229 29273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.548187 29269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:39.548470 29263 server_base.cc:1034] running on GCE node
I20260811 14:30:39.548804 29263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:39.549105 29263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:39.550316 29263 hybrid_clock.cc:648] HybridClock initialized: now 1786458639550294 us; error 48 us; skew 500 ppm
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:39.553476 29263 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:39.553939 29263 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:39.554003 29263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:39.554155 29263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:39.555498 29263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "de6b76c2e15d5f8b647651ba477468ff"
server_key_iv: "ab48fd98ff84fe6f297878154ee3f8de"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.555925 29263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "de6b76c2e15d5f8b647651ba477468ff"
server_key_iv: "ab48fd98ff84fe6f297878154ee3f8de"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.557886 29263 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 14:30:39.559029 29279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.559464 29263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:39.559574 29263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "de6b76c2e15d5f8b647651ba477468ff"
server_key_iv: "ab48fd98ff84fe6f297878154ee3f8de"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.559700 29263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:39.571224 29263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:39.573864 29263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:39.574046 29263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:39.574354 29263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:39.574782 29263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:39.574849 29263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.574887 29263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:39.574920 29263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.583207 29263 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:44771
I20260811 14:30:39.583453 29407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:44771 every 8 connection(s)
I20260811 14:30:39.583822 29263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:39.593829 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29263
I20260811 14:30:39.593976 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:39.594296 24299 external_mini_cluster.cc:1428] Setting key f4415ce8cb7775a14e5c7b906d5e42d5
I20260811 14:30:39.597071 29411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.584885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37065 (local address 127.23.186.194:60605)
0811 14:30:39.585109 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.585152 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.585702 (+ 550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.585981 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.585985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.586279 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:39.586853 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.586859 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.587634 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.587635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.589296 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.589302 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.589779 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.589780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.589946 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.590623 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:39.590634 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:39.592635 (+ 2001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:39.595050 (+ 2415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:39.595065 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:39.595069 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:39.595458 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:39.595789 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:39.595799 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:39.595799 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:39.595940 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:39.596364 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:39.596371 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:39.596525 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.596633 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.596875 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":100,"threads_started":1}
I20260811 14:30:39.597055 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.584877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:60605 (local address 127.23.186.254:37065)
0811 14:30:39.584976 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.584978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.585246 (+ 268us) server_negotiation.cc:407] Connection header received
0811 14:30:39.585822 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.585824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.585837 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.585900 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:39.586978 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.587523 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.589425 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.589664 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.589767 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.592879 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:39.592881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:39.592882 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:39.592892 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:39.594926 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.595590 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.595592 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.595592 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.595678 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.596041 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.596042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.596043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.596247 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:39.596339 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.596758 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.596887 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 14:30:39.597664 29408 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37065
I20260811 14:30:39.597800 29408 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:39.597980 29408 heartbeater.cc:511] Master 127.23.186.254:37065 requested a full tablet report, sending...
I20260811 14:30:39.598407 29035 ts_manager.cc:194] Registered new tserver with Master: 9af6fc8c73524d5ebe17bb3089e7e0fe (127.23.186.194:44771)
I20260811 14:30:39.598932 29035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:60605
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:39.629596 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37065
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:39.733274 29415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:39.733568 29415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:39.733610 29415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:39.735498 29415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:39.735621 29415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:39.735666 29415 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:39.738384 29415 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:37065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:39.738775 29415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:39.739133 29415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:39.745340 29422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.745340 29421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:39.745343 29425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:39.745707 29415 server_base.cc:1034] running on GCE node
I20260811 14:30:39.746014 29415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:39.746326 29415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:39.747524 29415 hybrid_clock.cc:648] HybridClock initialized: now 1786458639747505 us; error 39 us; skew 500 ppm
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:39.751587 29415 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:39.752213 29415 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:39.752313 29415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:39.752496 29415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:39.753873 29415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "d352f56a612043758d625e764b632de5"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "e21e8fc2a239ba48cbb4658ff364c3f6"
server_key_iv: "7893f902564b445c155c1b44c89b9324"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.754299 29415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "d352f56a612043758d625e764b632de5"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "e21e8fc2a239ba48cbb4658ff364c3f6"
server_key_iv: "7893f902564b445c155c1b44c89b9324"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.756012 29415 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:39.757550 29434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.757946 29415 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:39.758133 29415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "d352f56a612043758d625e764b632de5"
format_stamp: "Formatted at 2026-08-11 14:30:39 on dist-test-slave-cbsf"
server_key: "e21e8fc2a239ba48cbb4658ff364c3f6"
server_key_iv: "7893f902564b445c155c1b44c89b9324"
server_key_version: "encryptionkey@0"
I20260811 14:30:39.758239 29415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:39.769955 29415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:39.773339 29415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:39.773543 29415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:39.773839 29415 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:39.774230 29415 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:39.774360 29415 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.774400 29415 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:39.774413 29415 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:39.781183 29415 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:42967
I20260811 14:30:39.781265 29569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:42967 every 8 connection(s)
I20260811 14:30:39.781574 29415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:39.788944 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29415
I20260811 14:30:39.789038 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:39.789245 24299 external_mini_cluster.cc:1428] Setting key c834a5e888139062e19e4fa5d94ee9dc
Aug 11 14:30:39 dist-test-slave-cbsf krb5kdc[28986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458639, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:39.794056 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.783030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:45327 (local address 127.23.186.254:37065)
0811 14:30:39.783102 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.783104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.783314 (+ 210us) server_negotiation.cc:407] Connection header received
0811 14:30:39.783845 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.783847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.783858 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.783918 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:39.785133 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.785757 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.787022 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.787271 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.787354 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.790419 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:39.790421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:39.790421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:39.790430 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:39.792120 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.792804 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.792806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.792806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.792886 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.793190 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.793192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.793192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.793386 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:39.793472 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.793822 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.793927 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:30:39.794435 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.782990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37065 (local address 127.23.186.195:45327)
0811 14:30:39.783191 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.783232 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.783745 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.783989 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.783993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.784291 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:39.784973 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.784980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.785905 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.785906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.786881 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.786883 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.787391 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.787392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.787531 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.788108 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:39.788126 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:39.790300 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:39.792296 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:39.792314 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:39.792318 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:39.792664 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:39.792961 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:39.792962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:39.792962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:39.793113 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:39.793460 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:39.793468 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:39.793663 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.793796 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.794018 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":120,"threads_started":1}
I20260811 14:30:39.795133 29570 heartbeater.cc:347] Connected to a master server at 127.23.186.254:37065
I20260811 14:30:39.795290 29570 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:39.795496 29570 heartbeater.cc:511] Master 127.23.186.254:37065 requested a full tablet report, sending...
I20260811 14:30:39.796030 29035 ts_manager.cc:194] Registered new tserver with Master: d352f56a612043758d625e764b632de5 (127.23.186.195:42967)
I20260811 14:30:39.796484 29035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:45327
I20260811 14:30:39.802201 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:30:39.812855 29080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.804129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.23.186.254:37065)
0811 14:30:39.804317 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.804321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.804458 (+ 137us) server_negotiation.cc:407] Connection header received
0811 14:30:39.804574 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.804575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.804591 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.804679 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:39.805508 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.806124 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.807085 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.807330 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.807418 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.809242 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:39.809249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:39.809249 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:39.809260 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:39.811030 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.811552 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.811554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.811554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.811631 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:39.811868 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:39.811869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:39.811870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:39.812092 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:39.812331 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.812587 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.812736 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 14:30:39.815140 29035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:39.817539 29035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:39.825888 29581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.821059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:44771 (local address 127.0.0.1:35054)
0811 14:30:39.821397 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.821407 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.821451 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.822083 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.822084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.822087 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:39.822151 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.822153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.823617 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.823618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.824597 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.824600 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.825473 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.825474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.825580 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.825597 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.825711 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.825778 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":133,"threads_started":1}
I20260811 14:30:39.826247 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.821581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58010 (local address 127.23.186.193:35895)
0811 14:30:39.822211 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.822215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.822225 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:39.822264 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.822266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.822290 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.822391 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:39.822515 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.823340 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.824844 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.825593 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.825673 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.825721 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.825944 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.826157 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":101,"threads_started":1}
I20260811 14:30:39.826333 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.821205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.23.186.194:44771)
0811 14:30:39.821908 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.821913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.821922 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:39.821959 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.821962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.821977 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.822079 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:39.822250 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.823512 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.824733 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.825351 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.825472 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.825542 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.826185 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.826237 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":104,"threads_started":1}
I20260811 14:30:39.826689 29586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.821317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:35895 (local address 127.0.0.1:58010)
0811 14:30:39.821444 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.821461 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.821524 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.822389 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.822390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.822392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:39.822449 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.822451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.823440 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.823441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.824696 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.824700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.825686 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.825687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.825766 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.825770 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.825825 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.825865 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":53,"thread_start_us":77,"threads_started":1}
I20260811 14:30:39.827129 29334 tablet_service.cc:1467] Processing CreateTablet for tablet 70e2bfb7331d4a14be5f2162b41cfea2 (DEFAULT_TABLE table=test-table [id=13f2f35c7a3f4702821b8a7285fab280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:39.827632 29334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e2bfb7331d4a14be5f2162b41cfea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:39.828961 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.820993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:42967 (local address 127.0.0.1:33098)
0811 14:30:39.821254 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.821303 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.821380 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.821837 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.821840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.821842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:39.821950 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.821953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.823526 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.823528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.825756 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.825759 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.828507 (+ 2748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.828510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.828608 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.828614 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.828840 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.828903 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":1018,"thread_start_us":84,"threads_started":1}
I20260811 14:30:39.829012 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.821203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.23.186.195:42967)
0811 14:30:39.821601 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.821607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.821619 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:39.821665 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.821667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.821685 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.821769 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:39.822023 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.823396 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.825828 (+ 2432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.828320 (+ 2492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.828424 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.828483 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.828861 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.828915 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":112,"threads_started":1}
I20260811 14:30:39.829809 29492 tablet_service.cc:1467] Processing CreateTablet for tablet 70e2bfb7331d4a14be5f2162b41cfea2 (DEFAULT_TABLE table=test-table [id=13f2f35c7a3f4702821b8a7285fab280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:39.830180 29492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e2bfb7331d4a14be5f2162b41cfea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:39.831099 29588 tablet_bootstrap.cc:492] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Bootstrap starting.
I20260811 14:30:39.831851 29588 tablet_bootstrap.cc:654] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:39.832312 29181 tablet_service.cc:1467] Processing CreateTablet for tablet 70e2bfb7331d4a14be5f2162b41cfea2 (DEFAULT_TABLE table=test-table [id=13f2f35c7a3f4702821b8a7285fab280]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:39.832505 29588 log.cc:824] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:39.832760 29181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70e2bfb7331d4a14be5f2162b41cfea2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:39.833371 29590 tablet_bootstrap.cc:492] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Bootstrap starting.
I20260811 14:30:39.833609 29588 tablet_bootstrap.cc:492] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: No bootstrap required, opened a new log
I20260811 14:30:39.833695 29588 ts_tablet_manager.cc:1397] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:30:39.834041 29590 tablet_bootstrap.cc:654] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:39.834649 29590 log.cc:824] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:39.835400 29590 tablet_bootstrap.cc:492] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: No bootstrap required, opened a new log
I20260811 14:30:39.835491 29590 ts_tablet_manager.cc:1397] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:30:39.835870 29588 raft_consensus.cc:348] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.836068 29588 raft_consensus.cc:374] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:39.836126 29588 raft_consensus.cc:729] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af6fc8c73524d5ebe17bb3089e7e0fe, State: Initialized, Role: FOLLOWER
I20260811 14:30:39.836287 29588 consensus_queue.cc:260] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.836421 29592 tablet_bootstrap.cc:492] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: Bootstrap starting.
I20260811 14:30:39.836601 29588 ts_tablet_manager.cc:1428] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:30:39.836764 29408 heartbeater.cc:503] Master 127.23.186.254:37065 was elected leader, sending a full tablet report...
I20260811 14:30:39.837404 29590 raft_consensus.cc:348] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.837613 29590 raft_consensus.cc:374] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:39.837662 29590 raft_consensus.cc:729] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d352f56a612043758d625e764b632de5, State: Initialized, Role: FOLLOWER
I20260811 14:30:39.837685 29592 tablet_bootstrap.cc:654] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:39.837790 29590 consensus_queue.cc:260] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.838093 29590 ts_tablet_manager.cc:1428] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:39.838213 29570 heartbeater.cc:503] Master 127.23.186.254:37065 was elected leader, sending a full tablet report...
I20260811 14:30:39.838352 29592 log.cc:824] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:39.839172 29592 tablet_bootstrap.cc:492] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: No bootstrap required, opened a new log
I20260811 14:30:39.839241 29592 ts_tablet_manager.cc:1397] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:30:39.840811 29592 raft_consensus.cc:348] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.840953 29592 raft_consensus.cc:374] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:39.840974 29592 raft_consensus.cc:729] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f515d9d2baf649278bc3cdacbd7b49e4, State: Initialized, Role: FOLLOWER
I20260811 14:30:39.841099 29592 consensus_queue.cc:260] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.841424 29592 ts_tablet_manager.cc:1428] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:30:39.841639 29258 heartbeater.cc:503] Master 127.23.186.254:37065 was elected leader, sending a full tablet report...
W20260811 14:30:39.900436 29259 tablet.cc:2369] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:39.902522 29594 raft_consensus.cc:482] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:39.902678 29594 raft_consensus.cc:504] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.903082 29594 leader_election.cc:288] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9af6fc8c73524d5ebe17bb3089e7e0fe (127.23.186.194:44771), f515d9d2baf649278bc3cdacbd7b49e4 (127.23.186.193:35895)
I20260811 14:30:39.907735 29583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.903555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:51293 (local address 127.23.186.194:44771)
0811 14:30:39.903646 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.903649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.903660 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:39.903720 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.903721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.903732 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.903797 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:39.904110 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.904798 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.906400 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.907098 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.907211 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.907223 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.907603 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.907652 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 14:30:39.907735 29587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.903800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:54481 (local address 127.23.186.193:35895)
0811 14:30:39.903855 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.903857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.904015 (+ 158us) server_negotiation.cc:407] Connection header received
0811 14:30:39.904097 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.904098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.904108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.904176 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:39.904571 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.905162 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.906587 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.907227 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.907299 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.907309 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.907603 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.907652 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:30:39.907871 29572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.903428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:44771 (local address 127.23.186.195:51293)
0811 14:30:39.903522 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.903556 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.903650 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.903871 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.903873 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.903877 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:39.904008 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.904011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.904901 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.904903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.906260 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.906264 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.907320 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.907322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.907473 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.907485 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.907581 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.907750 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":8}
I20260811 14:30:39.908135 29597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.903601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:35895 (local address 127.23.186.195:54481)
0811 14:30:39.903902 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.903916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:39.904008 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:39.904191 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:39.904192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:39.904194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:39.904457 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.904464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.905268 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.905269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.906485 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:39.906487 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.907345 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.907346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:39.907473 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.907485 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 14:30:39.907581 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 14:30:39.907749 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":43,"thread_start_us":183,"threads_started":1}
I20260811 14:30:39.908342 29357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e2bfb7331d4a14be5f2162b41cfea2" candidate_uuid: "d352f56a612043758d625e764b632de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" is_pre_election: true
I20260811 14:30:39.908550 29357 raft_consensus.cc:2454] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d352f56a612043758d625e764b632de5 in term 0.
I20260811 14:30:39.908897 29435 leader_election.cc:302] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9af6fc8c73524d5ebe17bb3089e7e0fe, d352f56a612043758d625e764b632de5; no voters:
I20260811 14:30:39.908906 29204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e2bfb7331d4a14be5f2162b41cfea2" candidate_uuid: "d352f56a612043758d625e764b632de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" is_pre_election: true
I20260811 14:30:39.909076 29204 raft_consensus.cc:2454] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d352f56a612043758d625e764b632de5 in term 0.
I20260811 14:30:39.909054 29594 raft_consensus.cc:2781] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:39.909118 29594 raft_consensus.cc:482] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:39.909159 29594 raft_consensus.cc:3037] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:39.909982 29594 raft_consensus.cc:504] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.910142 29594 leader_election.cc:288] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [CANDIDATE]: Term 1 election: Requested vote from peers 9af6fc8c73524d5ebe17bb3089e7e0fe (127.23.186.194:44771), f515d9d2baf649278bc3cdacbd7b49e4 (127.23.186.193:35895)
I20260811 14:30:39.910303 29357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e2bfb7331d4a14be5f2162b41cfea2" candidate_uuid: "d352f56a612043758d625e764b632de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe"
I20260811 14:30:39.910435 29357 raft_consensus.cc:3037] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:39.910408 29204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70e2bfb7331d4a14be5f2162b41cfea2" candidate_uuid: "d352f56a612043758d625e764b632de5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f515d9d2baf649278bc3cdacbd7b49e4"
I20260811 14:30:39.910622 29204 raft_consensus.cc:3037] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:39.911312 29357 raft_consensus.cc:2454] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d352f56a612043758d625e764b632de5 in term 1.
I20260811 14:30:39.911584 29435 leader_election.cc:302] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9af6fc8c73524d5ebe17bb3089e7e0fe, d352f56a612043758d625e764b632de5; no voters:
I20260811 14:30:39.911737 29594 raft_consensus.cc:2781] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:39.911981 29594 raft_consensus.cc:686] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 1 LEADER]: Becoming Leader. State: Replica: d352f56a612043758d625e764b632de5, State: Running, Role: LEADER
I20260811 14:30:39.911991 29204 raft_consensus.cc:2454] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d352f56a612043758d625e764b632de5 in term 1.
I20260811 14:30:39.912276 29594 consensus_queue.cc:237] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } }
I20260811 14:30:39.913439 29035 catalog_manager.cc:5495] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d352f56a612043758d625e764b632de5 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "d352f56a612043758d625e764b632de5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d352f56a612043758d625e764b632de5" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42967 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:39.926012 29584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.922946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.23.186.195:42967)
0811 14:30:39.923108 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:39.923111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:39.923132 (+ 21us) server_negotiation.cc:407] Connection header received
0811 14:30:39.923178 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:39.923179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:39.923202 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:39.923284 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:39.923542 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.924137 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.925053 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:39.925269 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:39.925341 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:39.925613 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:39.925744 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:39.925834 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:39.925889 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 14:30:39.925911 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 14:30:39.928755 29204 raft_consensus.cc:1261] T 70e2bfb7331d4a14be5f2162b41cfea2 P f515d9d2baf649278bc3cdacbd7b49e4 [term 1 FOLLOWER]: Refusing update from remote peer d352f56a612043758d625e764b632de5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:39.928782 29357 raft_consensus.cc:1261] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 1 FOLLOWER]: Refusing update from remote peer d352f56a612043758d625e764b632de5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:39.929381 29598 consensus_queue.cc:1035] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f515d9d2baf649278bc3cdacbd7b49e4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 35895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:39.929600 29594 consensus_queue.cc:1035] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9af6fc8c73524d5ebe17bb3089e7e0fe" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 44771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:39.932924 29605 mvcc.cc:205] Tried to move back new op lower bound from 7317334589143265280 to 7317334589081870336. Current Snapshot: MvccSnapshot[applied={T|T < 7317334589143265280}]
I20260811 14:30:39.933404 29603 mvcc.cc:205] Tried to move back new op lower bound from 7317334589143265280 to 7317334589081870336. Current Snapshot: MvccSnapshot[applied={T|T < 7317334589143265280}]
I20260811 14:30:39.935654 29034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:39.935758 29034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:39.936867 29034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4884c3b42cf1421c9fad01acab212944: Sending DeleteTablet for 3 replicas of tablet 70e2bfb7331d4a14be5f2162b41cfea2
I20260811 14:30:39.937381 29492 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e2bfb7331d4a14be5f2162b41cfea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:39 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:33098
I20260811 14:30:39.937580 29334 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e2bfb7331d4a14be5f2162b41cfea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:39 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:35054
I20260811 14:30:39.937592 29181 tablet_service.cc:1514] Processing DeleteTablet for tablet 70e2bfb7331d4a14be5f2162b41cfea2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:39 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:58010
I20260811 14:30:39.937844 29615 tablet_replica.cc:330] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: stopping tablet replica
I20260811 14:30:39.937968 29615 raft_consensus.cc:2229] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:39.938102 29615 raft_consensus.cc:2258] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:39.938141 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29106
I20260811 14:30:39.942490 29615 ts_tablet_manager.cc:1905] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:39.944936 29615 ts_tablet_manager.cc:1918] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:39.945039 29615 log.cc:1196] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/70e2bfb7331d4a14be5f2162b41cfea2
I20260811 14:30:39.945369 29615 ts_tablet_manager.cc:1939] T 70e2bfb7331d4a14be5f2162b41cfea2 P 9af6fc8c73524d5ebe17bb3089e7e0fe: Deleting consensus metadata
I20260811 14:30:39.945973 29017 catalog_manager.cc:4841] TS 9af6fc8c73524d5ebe17bb3089e7e0fe (127.23.186.194:44771): tablet 70e2bfb7331d4a14be5f2162b41cfea2 (table test-table [id=13f2f35c7a3f4702821b8a7285fab280]) successfully deleted
I20260811 14:30:39.948923 29614 tablet_replica.cc:330] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: stopping tablet replica
I20260811 14:30:39.949111 29614 raft_consensus.cc:2229] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:39.949317 29614 raft_consensus.cc:2258] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:39.949884 29614 ts_tablet_manager.cc:1905] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 14:30:39.951324 29017 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:35895 (error 108)
I20260811 14:30:39.951576 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29263
I20260811 14:30:39.951972 29582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:39.951828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:35895 (local address 127.0.0.1:58022)
0811 14:30:39.951883 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:39.951927 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:35895: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260811 14:30:39.952102 29017 catalog_manager.cc:4584] TS f515d9d2baf649278bc3cdacbd7b49e4 (127.23.186.193:35895): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 70e2bfb7331d4a14be5f2162b41cfea2: Network error: Client connection negotiation failed: client connection to 127.23.186.193:35895: connect: Connection refused (error 111)
I20260811 14:30:39.953286 29614 ts_tablet_manager.cc:1918] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:39.953377 29614 log.cc:1196] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/70e2bfb7331d4a14be5f2162b41cfea2
I20260811 14:30:39.953666 29614 ts_tablet_manager.cc:1939] T 70e2bfb7331d4a14be5f2162b41cfea2 P d352f56a612043758d625e764b632de5: Deleting consensus metadata
I20260811 14:30:39.954203 29018 catalog_manager.cc:4841] TS d352f56a612043758d625e764b632de5 (127.23.186.195:42967): tablet 70e2bfb7331d4a14be5f2162b41cfea2 (table test-table [id=13f2f35c7a3f4702821b8a7285fab280]) successfully deleted
I20260811 14:30:39.963218 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29415
I20260811 14:30:39.974978 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29002
2026-08-11T14:30:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2224 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T14:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:40Z Disabled control of system clock
I20260811 14:30:40.015082 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38351
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:30:40.107935 29622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:40.108186 29622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:40.108209 29622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:40.109423 29622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:40.109469 29622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:40.109483 29622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:40.109495 29622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:40.109508 29622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:40.110898 29622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:38351
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:40.111153 29622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:40.111438 29622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:40.116788 29629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.116782 29632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.116782 29627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:40.117336 29622 server_base.cc:1034] running on GCE node
I20260811 14:30:40.117652 29622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:40.117980 29622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:40.119176 29622 hybrid_clock.cc:648] HybridClock initialized: now 1786458640119162 us; error 53 us; skew 500 ppm
I20260811 14:30:40.120667 29622 webserver.cc:460] Webserver started at http://127.23.186.254:35405/ using document root <none> and password file <none>
I20260811 14:30:40.120960 29622 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:40.121011 29622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:40.121124 29622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:40.122262 29622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "2cfa62ca16834e46ae73244bb48bdb19"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "62a76fdea5bf5a08bc57286cf761fd79"
server_key_iv: "76a07fcaf34fd26ecb466e87fa16c96c"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.122823 29622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "2cfa62ca16834e46ae73244bb48bdb19"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "62a76fdea5bf5a08bc57286cf761fd79"
server_key_iv: "76a07fcaf34fd26ecb466e87fa16c96c"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.124321 29622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:40.125365 29639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.125737 29622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:30:40.125859 29622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "2cfa62ca16834e46ae73244bb48bdb19"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "62a76fdea5bf5a08bc57286cf761fd79"
server_key_iv: "76a07fcaf34fd26ecb466e87fa16c96c"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.125929 29622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:40.129594 29622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:40.129853 29622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:40.129962 29622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:40.134183 29622 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:38351
I20260811 14:30:40.134220 29697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:38351 every 8 connection(s)
I20260811 14:30:40.134619 29622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:40.135462 29700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:40.138545 29700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19: Bootstrap starting.
I20260811 14:30:40.139204 29700 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:40.139523 29700 log.cc:824] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:40.140305 29700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19: No bootstrap required, opened a new log
I20260811 14:30:40.142082 29700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfa62ca16834e46ae73244bb48bdb19" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38351 } }
I20260811 14:30:40.142237 29700 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:40.142261 29700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cfa62ca16834e46ae73244bb48bdb19, State: Initialized, Role: FOLLOWER
I20260811 14:30:40.142428 29700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfa62ca16834e46ae73244bb48bdb19" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38351 } }
I20260811 14:30:40.142511 29700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:40.142535 29700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:40.142608 29700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:40.143092 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29622
I20260811 14:30:40.143190 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:40.143308 29700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfa62ca16834e46ae73244bb48bdb19" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38351 } }
I20260811 14:30:40.143436 29700 leader_election.cc:302] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cfa62ca16834e46ae73244bb48bdb19; no voters:
I20260811 14:30:40.143474 24299 external_mini_cluster.cc:1428] Setting key 488d45f48f957022967d0246dd4bd753
I20260811 14:30:40.143671 29700 leader_election.cc:288] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:40.143702 29703 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:40.143915 29703 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [term 1 LEADER]: Becoming Leader. State: Replica: 2cfa62ca16834e46ae73244bb48bdb19, State: Running, Role: LEADER
I20260811 14:30:40.143985 29700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:40.144085 29703 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfa62ca16834e46ae73244bb48bdb19" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38351 } }
I20260811 14:30:40.144462 29703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cfa62ca16834e46ae73244bb48bdb19. Latest consensus state: current_term: 1 leader_uuid: "2cfa62ca16834e46ae73244bb48bdb19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfa62ca16834e46ae73244bb48bdb19" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38351 } } }
I20260811 14:30:40.144541 29703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:40.144793 29710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:40.144963 29704 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cfa62ca16834e46ae73244bb48bdb19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cfa62ca16834e46ae73244bb48bdb19" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38351 } } }
I20260811 14:30:40.145140 29704 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:40.145574 29710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:40.147542 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.144320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36108 (local address 127.23.186.254:38351)
0811 14:30:40.145612 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.145622 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.145665 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:30:40.145897 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.145906 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.145966 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.146123 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.147011 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.147015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.147015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.147021 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.147072 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.147088 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.147169 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.147254 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.147318 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1202,"thread_start_us":70,"threads_started":1}
I20260811 14:30:40.147557 29710 catalog_manager.cc:1347] Generated new cluster ID: 2c9d92cebe444be1a74356ed55a2b584
I20260811 14:30:40.147657 29710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:40.158970 29710 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:40.159201 29710 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:40.165360 29710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19: Generated new TSK 0
I20260811 14:30:40.165609 29710 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:30:40.170339 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38351
--builtin_ntp_servers=127.23.186.212:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:30:40.261811 29727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:40.262097 29727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:40.262145 29727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:40.263461 29727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:40.263537 29727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:40.263592 29727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:40.265189 29727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:40.265496 29727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:40.265844 29727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:40.271562 29737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.271553 29735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.271684 29733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:40.271998 29727 server_base.cc:1034] running on GCE node
I20260811 14:30:40.272226 29727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:40.272501 29727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:40.273694 29727 hybrid_clock.cc:648] HybridClock initialized: now 1786458640273670 us; error 44 us; skew 500 ppm
I20260811 14:30:40.275200 29727 webserver.cc:460] Webserver started at http://127.23.186.193:41609/ using document root <none> and password file <none>
I20260811 14:30:40.275511 29727 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:40.275586 29727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:40.275761 29727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:40.276944 29727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "19b2475310b0407a8bdf1dbae159dfb0"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "6e805c2acd18bde7904740e927990720"
server_key_iv: "c5d50a1f56799e84f87dce7b31cb45c5"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.277349 29727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "19b2475310b0407a8bdf1dbae159dfb0"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "6e805c2acd18bde7904740e927990720"
server_key_iv: "c5d50a1f56799e84f87dce7b31cb45c5"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.278801 29727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:40.279677 29743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.279847 29727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:40.279960 29727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "19b2475310b0407a8bdf1dbae159dfb0"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "6e805c2acd18bde7904740e927990720"
server_key_iv: "c5d50a1f56799e84f87dce7b31cb45c5"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.280041 29727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:40.283728 29727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:40.284042 29727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:40.284195 29727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:40.284420 29727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:40.284811 29727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:40.284874 29727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.284915 29727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:40.284942 29727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.291172 29727 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:46343
I20260811 14:30:40.291222 29873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:46343 every 8 connection(s)
I20260811 14:30:40.291589 29727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:40.294962 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.292367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:48057 (local address 127.23.186.254:38351)
0811 14:30:40.292457 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.292459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.292656 (+ 197us) server_negotiation.cc:407] Connection header received
0811 14:30:40.293238 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.293239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.293248 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.293308 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.294610 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.294612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.294612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.294617 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.294635 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.294639 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.294696 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.294872 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.294891 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 14:30:40.295001 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.292314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38351 (local address 127.23.186.193:48057)
0811 14:30:40.292530 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.292568 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.293114 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.293363 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.293366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.293375 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.294410 (+ 1035us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.294410 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.294443 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.294476 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.294694 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.294697 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.294699 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.294772 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.294826 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":108,"threads_started":1}
I20260811 14:30:40.295591 29874 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38351
I20260811 14:30:40.295687 29874 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:40.295869 29874 heartbeater.cc:511] Master 127.23.186.254:38351 requested a full tablet report, sending...
I20260811 14:30:40.296445 29660 ts_manager.cc:194] Registered new tserver with Master: 19b2475310b0407a8bdf1dbae159dfb0 (127.23.186.193:46343)
I20260811 14:30:40.298077 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29727
I20260811 14:30:40.298170 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:40.298381 24299 external_mini_cluster.cc:1428] Setting key 44aa7600e73297cdba6d6ac30db32d0a
I20260811 14:30:40.299672 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38351
--builtin_ntp_servers=127.23.186.212:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:30:40.393226 29877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:40.393500 29877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:40.393553 29877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:40.394861 29877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:40.394939 29877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:40.394986 29877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:40.396514 29877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:40.396864 29877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:40.397233 29877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:40.402810 29884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.402791 29885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.402781 29887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:40.403136 29877 server_base.cc:1034] running on GCE node
I20260811 14:30:40.403358 29877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:40.403650 29877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:40.404785 29877 hybrid_clock.cc:648] HybridClock initialized: now 1786458640404754 us; error 39 us; skew 500 ppm
I20260811 14:30:40.406219 29877 webserver.cc:460] Webserver started at http://127.23.186.194:36607/ using document root <none> and password file <none>
I20260811 14:30:40.406600 29877 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:40.406656 29877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:40.406803 29877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:40.408011 29877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "89bbbb71bd494e1bbcc7cb38cdc74474"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "24431d1309a9198e6bff34735ac36edf"
server_key_iv: "577603a308ddfc6df59ed34cf7b61a28"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.408435 29877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "89bbbb71bd494e1bbcc7cb38cdc74474"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "24431d1309a9198e6bff34735ac36edf"
server_key_iv: "577603a308ddfc6df59ed34cf7b61a28"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.409979 29877 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:40.411082 29895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.411475 29877 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:40.411586 29877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "89bbbb71bd494e1bbcc7cb38cdc74474"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "24431d1309a9198e6bff34735ac36edf"
server_key_iv: "577603a308ddfc6df59ed34cf7b61a28"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.411676 29877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:40.415509 29877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:40.415822 29877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:40.415987 29877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:40.416232 29877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:40.416589 29877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:40.416651 29877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.416718 29877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:40.416747 29877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.423444 29877 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:36139
I20260811 14:30:40.423496 30024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:36139 every 8 connection(s)
I20260811 14:30:40.423873 29877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:40.427222 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.424870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:41121 (local address 127.23.186.254:38351)
0811 14:30:40.424960 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.424963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.425119 (+ 156us) server_negotiation.cc:407] Connection header received
0811 14:30:40.425585 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.425587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.425595 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.425674 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.426914 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.426915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.426916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.426920 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.426938 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.426942 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.427001 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.427145 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.427160 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 14:30:40.427430 30028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.424657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38351 (local address 127.23.186.194:41121)
0811 14:30:40.424932 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.424985 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.425479 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.425746 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.425749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.425758 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.426749 (+ 991us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.426749 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.426782 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.426816 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.427048 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.427056 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.427059 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.427144 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.427202 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":165,"threads_started":1}
I20260811 14:30:40.427798 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 29877
I20260811 14:30:40.427910 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:40.428045 30025 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38351
I20260811 14:30:40.428125 24299 external_mini_cluster.cc:1428] Setting key 0e693739238333a441d51e5970e944f5
I20260811 14:30:40.428169 30025 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:40.428340 30025 heartbeater.cc:511] Master 127.23.186.254:38351 requested a full tablet report, sending...
I20260811 14:30:40.428846 29659 ts_manager.cc:194] Registered new tserver with Master: 89bbbb71bd494e1bbcc7cb38cdc74474 (127.23.186.194:36139)
I20260811 14:30:40.429250 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38351
--builtin_ntp_servers=127.23.186.212:35441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 14:30:40.520872 30029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:40.521128 30029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:40.521157 30029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:40.522392 30029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:40.522441 30029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:40.522470 30029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:40.523849 30029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:40.524088 30029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:40.524471 30029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:40.529984 30039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.530038 30038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.530006 30041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:40.530369 30029 server_base.cc:1034] running on GCE node
I20260811 14:30:40.530586 30029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:40.530854 30029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:40.532051 30029 hybrid_clock.cc:648] HybridClock initialized: now 1786458640531956 us; error 103 us; skew 500 ppm
I20260811 14:30:40.533679 30029 webserver.cc:460] Webserver started at http://127.23.186.195:35031/ using document root <none> and password file <none>
I20260811 14:30:40.533962 30029 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:40.534111 30029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:40.534338 30029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:40.535640 30029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "1edb5f954a7a47419eec2bcaeceb4495"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "0fd7aefc04969f65f161f71431b6fced"
server_key_iv: "f4120744cc3f416ab2d623d06d6f5d90"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.536049 30029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "1edb5f954a7a47419eec2bcaeceb4495"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "0fd7aefc04969f65f161f71431b6fced"
server_key_iv: "f4120744cc3f416ab2d623d06d6f5d90"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.537668 30029 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260811 14:30:40.538579 30048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.538861 30029 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:40.539000 30029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "1edb5f954a7a47419eec2bcaeceb4495"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "0fd7aefc04969f65f161f71431b6fced"
server_key_iv: "f4120744cc3f416ab2d623d06d6f5d90"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.539112 30029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:40.543284 30029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:40.543720 30029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:40.543942 30029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:40.544216 30029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:40.544682 30029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:40.544801 30029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.544840 30029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:40.544873 30029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.551738 30029 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:34145
I20260811 14:30:40.551811 30179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:34145 every 8 connection(s)
I20260811 14:30:40.552237 30029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:40.555868 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.553148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:38749 (local address 127.23.186.254:38351)
0811 14:30:40.553200 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.553203 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.553476 (+ 273us) server_negotiation.cc:407] Connection header received
0811 14:30:40.554063 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.554064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.554081 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.554146 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.555547 (+ 1401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.555550 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.555550 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.555556 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.555574 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.555579 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.555647 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.555764 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.555784 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:40.555989 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30029
I20260811 14:30:40.556107 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:40.556102 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.553115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38351 (local address 127.23.186.195:38749)
0811 14:30:40.553342 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.553380 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.553938 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.554224 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.554229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.554241 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.555364 (+ 1123us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.555364 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.555397 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.555432 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.555661 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.555667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.555671 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.555776 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.555850 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
I20260811 14:30:40.556310 24299 external_mini_cluster.cc:1428] Setting key 25fd84d62ebcb54fdb4bdd3e1b9cd6c7
I20260811 14:30:40.556833 30180 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38351
I20260811 14:30:40.556954 30180 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:40.557150 30180 heartbeater.cc:511] Master 127.23.186.254:38351 requested a full tablet report, sending...
I20260811 14:30:40.557564 29659 ts_manager.cc:194] Registered new tserver with Master: 1edb5f954a7a47419eec2bcaeceb4495 (127.23.186.195:34145)
I20260811 14:30:40.568518 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:30:40.572052 29708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.570630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37370 (local address 127.23.186.254:38351)
0811 14:30:40.570716 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.570718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.570883 (+ 165us) server_negotiation.cc:407] Connection header received
0811 14:30:40.570950 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.570951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.570956 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.571007 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.571723 (+ 716us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.571724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.571725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.571729 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.571747 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.571751 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.571796 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.571893 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.571915 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 14:30:40.574110 29659 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:40.575773 29659 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:40.581058 30191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.578627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:36139 (local address 127.0.0.1:38892)
0811 14:30:40.579060 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.579070 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.579117 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.579301 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.579308 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.579317 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.579906 (+ 589us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.579907 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.579925 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.579955 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.580640 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.580659 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.580665 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.580912 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.580939 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":2,"spinlock_wait_cycles":384,"thread_start_us":117,"threads_started":1}
I20260811 14:30:40.581393 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.578820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38892 (local address 127.23.186.194:36139)
0811 14:30:40.579046 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.579050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.579115 (+ 65us) server_negotiation.cc:407] Connection header received
0811 14:30:40.579188 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.579192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.579203 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.579279 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.580316 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.580338 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.580339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.580348 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.580427 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.580482 (+ 55us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.580625 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.581259 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.581284 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":92,"threads_started":1}
I20260811 14:30:40.581668 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.578947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.23.186.195:34145)
0811 14:30:40.579189 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.579194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.579203 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:40.579237 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.579239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.579247 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.579365 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.581248 (+ 1883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.581261 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.581261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.581269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.581312 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.581344 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.581413 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.581546 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.581565 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":109,"threads_started":1}
I20260811 14:30:40.581741 30190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.578627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46343 (local address 127.0.0.1:33348)
0811 14:30:40.578853 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.578899 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.578964 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.580097 (+ 1133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.580098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.580101 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.580526 (+ 425us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.580526 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.580533 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.580545 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.581042 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.581044 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.581045 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.581106 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.581126 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":113,"threads_started":1}
I20260811 14:30:40.581874 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.578630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:34145 (local address 127.0.0.1:56778)
0811 14:30:40.579117 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.579128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.579179 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.580530 (+ 1351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.580531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.580533 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.581121 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.581121 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.581127 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.581137 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.581442 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.581443 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.581444 (+ 1us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.581492 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.581505 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":2,"thread_start_us":96,"threads_started":1}
I20260811 14:30:40.582551 29949 tablet_service.cc:1467] Processing CreateTablet for tablet 639ee2385c934e89a03b5a98a0c91f80 (DEFAULT_TABLE table=test-table [id=dbc50456fa2d491bb81092a454dbfb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:40.582888 30102 tablet_service.cc:1467] Processing CreateTablet for tablet 639ee2385c934e89a03b5a98a0c91f80 (DEFAULT_TABLE table=test-table [id=dbc50456fa2d491bb81092a454dbfb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:40.583050 29949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639ee2385c934e89a03b5a98a0c91f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:40.583271 30102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639ee2385c934e89a03b5a98a0c91f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:40.585209 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.578939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33348 (local address 127.23.186.193:46343)
0811 14:30:40.579937 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.579941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.579953 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:40.579981 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.579983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.579992 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.580096 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.580893 (+ 797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.580898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.580899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.580903 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.580936 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.580952 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.581051 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.585067 (+ 4016us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.585108 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":87,"threads_started":1}
I20260811 14:30:40.585916 29801 tablet_service.cc:1467] Processing CreateTablet for tablet 639ee2385c934e89a03b5a98a0c91f80 (DEFAULT_TABLE table=test-table [id=dbc50456fa2d491bb81092a454dbfb2d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:40.586274 29801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 639ee2385c934e89a03b5a98a0c91f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:40.587025 30196 tablet_bootstrap.cc:492] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Bootstrap starting.
I20260811 14:30:40.587735 30196 tablet_bootstrap.cc:654] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:40.588074 30196 log.cc:824] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:40.588517 30195 tablet_bootstrap.cc:492] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Bootstrap starting.
I20260811 14:30:40.589161 30196 tablet_bootstrap.cc:492] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: No bootstrap required, opened a new log
I20260811 14:30:40.589277 30195 tablet_bootstrap.cc:654] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:40.589375 30196 ts_tablet_manager.cc:1397] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 14:30:40.589649 30195 log.cc:824] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:40.590751 30195 tablet_bootstrap.cc:492] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: No bootstrap required, opened a new log
I20260811 14:30:40.590845 30195 ts_tablet_manager.cc:1397] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:30:40.591143 30199 tablet_bootstrap.cc:492] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0: Bootstrap starting.
I20260811 14:30:40.591792 30199 tablet_bootstrap.cc:654] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:40.592118 30199 log.cc:824] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:40.592578 30196 raft_consensus.cc:348] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.592816 30196 raft_consensus.cc:374] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:40.592866 30196 raft_consensus.cc:729] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1edb5f954a7a47419eec2bcaeceb4495, State: Initialized, Role: FOLLOWER
I20260811 14:30:40.593044 30196 consensus_queue.cc:260] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.593290 30199 tablet_bootstrap.cc:492] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0: No bootstrap required, opened a new log
I20260811 14:30:40.593379 30199 ts_tablet_manager.cc:1397] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:40.593417 30196 ts_tablet_manager.cc:1428] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260811 14:30:40.593590 30180 heartbeater.cc:503] Master 127.23.186.254:38351 was elected leader, sending a full tablet report...
I20260811 14:30:40.593748 30195 raft_consensus.cc:348] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.593907 30195 raft_consensus.cc:374] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:40.593942 30195 raft_consensus.cc:729] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89bbbb71bd494e1bbcc7cb38cdc74474, State: Initialized, Role: FOLLOWER
I20260811 14:30:40.594075 30195 consensus_queue.cc:260] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.594348 30195 ts_tablet_manager.cc:1428] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Time spent starting tablet: real 0.003s user 0.001s sys 0.003s
I20260811 14:30:40.594444 30025 heartbeater.cc:503] Master 127.23.186.254:38351 was elected leader, sending a full tablet report...
I20260811 14:30:40.596913 30199 raft_consensus.cc:348] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.597093 30199 raft_consensus.cc:374] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:40.597157 30199 raft_consensus.cc:729] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b2475310b0407a8bdf1dbae159dfb0, State: Initialized, Role: FOLLOWER
I20260811 14:30:40.597293 30199 consensus_queue.cc:260] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.597637 30199 ts_tablet_manager.cc:1428] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 14:30:40.597637 29874 heartbeater.cc:503] Master 127.23.186.254:38351 was elected leader, sending a full tablet report...
W20260811 14:30:40.674649 30027 tablet.cc:2369] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:40.747802 30201 raft_consensus.cc:482] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:40.747952 30201 raft_consensus.cc:504] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.748384 30201 leader_election.cc:288] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 89bbbb71bd494e1bbcc7cb38cdc74474 (127.23.186.194:36139), 19b2475310b0407a8bdf1dbae159dfb0 (127.23.186.193:46343)
I20260811 14:30:40.749965 30183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.748637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:36139 (local address 127.23.186.195:45193)
0811 14:30:40.748746 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.748761 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.748827 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.748982 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.748984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.748987 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.749505 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.749506 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.749515 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.749531 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.749792 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.749793 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.749795 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.749857 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.749874 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 14:30:40.750000 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.748773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:45193 (local address 127.23.186.194:36139)
0811 14:30:40.748828 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.748831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.748844 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:30:40.748886 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.748887 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.748893 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.748955 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.749661 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.749662 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.749663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.749670 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.749690 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.749696 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.749767 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.749869 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.749883 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:30:40.750082 30193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.748773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:41327 (local address 127.23.186.193:46343)
0811 14:30:40.748821 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.748823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.748962 (+ 139us) server_negotiation.cc:407] Connection header received
0811 14:30:40.748989 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.748991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.748997 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.749062 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.749832 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.749834 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.749834 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.749840 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.749858 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.749864 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.749921 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.749985 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.750000 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:40.750093 30204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.748637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46343 (local address 127.23.186.195:41327)
0811 14:30:40.748861 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.748871 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:40.748930 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:40.749085 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:40.749086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:40.749088 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:40.749707 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:30:40.749707 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:40.749715 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:30:40.749728 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:40.749930 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:40.749931 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:40.749931 (+ 0us) client_negotiation.cc:769] Sending connection context
0811 14:30:40.749984 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:30:40.749998 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":24,"thread_start_us":132,"threads_started":1}
I20260811 14:30:40.750316 29970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639ee2385c934e89a03b5a98a0c91f80" candidate_uuid: "1edb5f954a7a47419eec2bcaeceb4495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" is_pre_election: true
I20260811 14:30:40.750396 29824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639ee2385c934e89a03b5a98a0c91f80" candidate_uuid: "1edb5f954a7a47419eec2bcaeceb4495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b2475310b0407a8bdf1dbae159dfb0" is_pre_election: true
I20260811 14:30:40.750502 29970 raft_consensus.cc:2454] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edb5f954a7a47419eec2bcaeceb4495 in term 0.
I20260811 14:30:40.750521 29824 raft_consensus.cc:2454] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1edb5f954a7a47419eec2bcaeceb4495 in term 0.
I20260811 14:30:40.750787 30051 leader_election.cc:302] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b2475310b0407a8bdf1dbae159dfb0, 1edb5f954a7a47419eec2bcaeceb4495; no voters:
I20260811 14:30:40.750949 30201 raft_consensus.cc:2781] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:40.751015 30201 raft_consensus.cc:482] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:40.751031 30201 raft_consensus.cc:3037] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:40.751813 30201 raft_consensus.cc:504] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.751971 30201 leader_election.cc:288] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [CANDIDATE]: Term 1 election: Requested vote from peers 89bbbb71bd494e1bbcc7cb38cdc74474 (127.23.186.194:36139), 19b2475310b0407a8bdf1dbae159dfb0 (127.23.186.193:46343)
I20260811 14:30:40.752154 29824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639ee2385c934e89a03b5a98a0c91f80" candidate_uuid: "1edb5f954a7a47419eec2bcaeceb4495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b2475310b0407a8bdf1dbae159dfb0"
I20260811 14:30:40.752170 29970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "639ee2385c934e89a03b5a98a0c91f80" candidate_uuid: "1edb5f954a7a47419eec2bcaeceb4495" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474"
I20260811 14:30:40.752246 29970 raft_consensus.cc:3037] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:40.752246 29824 raft_consensus.cc:3037] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:40.753228 29970 raft_consensus.cc:2454] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edb5f954a7a47419eec2bcaeceb4495 in term 1.
I20260811 14:30:40.753228 29824 raft_consensus.cc:2454] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1edb5f954a7a47419eec2bcaeceb4495 in term 1.
I20260811 14:30:40.753443 30051 leader_election.cc:302] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19b2475310b0407a8bdf1dbae159dfb0, 1edb5f954a7a47419eec2bcaeceb4495; no voters:
I20260811 14:30:40.753575 30201 raft_consensus.cc:2781] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:40.753719 30201 raft_consensus.cc:686] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 1 LEADER]: Becoming Leader. State: Replica: 1edb5f954a7a47419eec2bcaeceb4495, State: Running, Role: LEADER
I20260811 14:30:40.753824 30201 consensus_queue.cc:237] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } }
I20260811 14:30:40.754796 29659 catalog_manager.cc:5495] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1edb5f954a7a47419eec2bcaeceb4495 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "1edb5f954a7a47419eec2bcaeceb4495" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1edb5f954a7a47419eec2bcaeceb4495" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:40.765482 30194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.764445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56794 (local address 127.23.186.195:34145)
0811 14:30:40.764499 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.764502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.764511 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:40.764564 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.764565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.764577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.764645 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.765242 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.765243 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.765244 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.765248 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.765264 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.765274 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.765335 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.765385 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.765402 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:30:40.767412 29824 raft_consensus.cc:1261] T 639ee2385c934e89a03b5a98a0c91f80 P 19b2475310b0407a8bdf1dbae159dfb0 [term 1 FOLLOWER]: Refusing update from remote peer 1edb5f954a7a47419eec2bcaeceb4495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:40.767413 29970 raft_consensus.cc:1261] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 1 FOLLOWER]: Refusing update from remote peer 1edb5f954a7a47419eec2bcaeceb4495: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:40.767841 30205 consensus_queue.cc:1035] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89bbbb71bd494e1bbcc7cb38cdc74474" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:40.767983 30201 consensus_queue.cc:1035] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b2475310b0407a8bdf1dbae159dfb0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:40.770720 30210 mvcc.cc:205] Tried to move back new op lower bound from 7317334592580202496 to 7317334592528179200. Current Snapshot: MvccSnapshot[applied={T|T < 7317334592580202496}]
I20260811 14:30:40.770944 30209 mvcc.cc:205] Tried to move back new op lower bound from 7317334592580202496 to 7317334592528179200. Current Snapshot: MvccSnapshot[applied={T|T < 7317334592580202496}]
I20260811 14:30:40.773114 29660 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:40.773201 29660 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:40.774025 29660 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2cfa62ca16834e46ae73244bb48bdb19: Sending DeleteTablet for 3 replicas of tablet 639ee2385c934e89a03b5a98a0c91f80
I20260811 14:30:40.774389 29949 tablet_service.cc:1514] Processing DeleteTablet for tablet 639ee2385c934e89a03b5a98a0c91f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:40 UTC) from {username='slave'} at 127.0.0.1:38892
I20260811 14:30:40.774389 30102 tablet_service.cc:1514] Processing DeleteTablet for tablet 639ee2385c934e89a03b5a98a0c91f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:40 UTC) from {username='slave'} at 127.0.0.1:56778
I20260811 14:30:40.774461 29801 tablet_service.cc:1514] Processing DeleteTablet for tablet 639ee2385c934e89a03b5a98a0c91f80 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:40 UTC) from {username='slave'} at 127.0.0.1:33348
I20260811 14:30:40.774653 30218 tablet_replica.cc:330] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: stopping tablet replica
I20260811 14:30:40.774760 30218 raft_consensus.cc:2229] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:40.774870 30218 raft_consensus.cc:2258] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:40.774952 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29727
I20260811 14:30:40.778187 30219 tablet_replica.cc:330] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: stopping tablet replica
I20260811 14:30:40.778308 30219 raft_consensus.cc:2229] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:40.778460 30219 raft_consensus.cc:2258] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:40.778600 30218 ts_tablet_manager.cc:1905] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:40.778942 30219 ts_tablet_manager.cc:1905] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:40.780761 30218 ts_tablet_manager.cc:1918] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:40.780761 30219 ts_tablet_manager.cc:1918] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:40.780851 30219 log.cc:1196] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/639ee2385c934e89a03b5a98a0c91f80
I20260811 14:30:40.780881 30218 log.cc:1196] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/639ee2385c934e89a03b5a98a0c91f80
I20260811 14:30:40.781172 30219 ts_tablet_manager.cc:1939] T 639ee2385c934e89a03b5a98a0c91f80 P 1edb5f954a7a47419eec2bcaeceb4495: Deleting consensus metadata
I20260811 14:30:40.781205 30218 ts_tablet_manager.cc:1939] T 639ee2385c934e89a03b5a98a0c91f80 P 89bbbb71bd494e1bbcc7cb38cdc74474: Deleting consensus metadata
I20260811 14:30:40.781622 29640 catalog_manager.cc:4841] TS 1edb5f954a7a47419eec2bcaeceb4495 (127.23.186.195:34145): tablet 639ee2385c934e89a03b5a98a0c91f80 (table test-table [id=dbc50456fa2d491bb81092a454dbfb2d]) successfully deleted
I20260811 14:30:40.781751 29642 catalog_manager.cc:4841] TS 89bbbb71bd494e1bbcc7cb38cdc74474 (127.23.186.194:36139): tablet 639ee2385c934e89a03b5a98a0c91f80 (table test-table [id=dbc50456fa2d491bb81092a454dbfb2d]) successfully deleted
W20260811 14:30:40.788760 29643 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:46343 (error 108)
I20260811 14:30:40.789390 30189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.789227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46343 (local address 127.0.0.1:33360)
0811 14:30:40.789299 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:40.789350 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:46343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 14:30:40.789492 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29877
W20260811 14:30:40.796444 29643 catalog_manager.cc:4584] TS 19b2475310b0407a8bdf1dbae159dfb0 (127.23.186.193:46343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 639ee2385c934e89a03b5a98a0c91f80: Network error: Client connection negotiation failed: client connection to 127.23.186.193:46343: connect: Connection refused (error 111)
I20260811 14:30:40.805419 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30029
I20260811 14:30:40.815001 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 29622
2026-08-11T14:30:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (836 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 14:30:40.838347 24299 mini_oidc.cc:152] Starting JWKS server
I20260811 14:30:40.839709 24299 webserver.cc:460] Webserver started at https://127.0.0.1:40389/ using document root <none> and password file <none>
I20260811 14:30:40.839797 24299 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 14:30:40.840008 24299 webserver.cc:460] Webserver started at http://127.0.0.1:41487/ using document root <none> and password file <none>
2026-08-11T14:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:40Z Disabled control of system clock
I20260811 14:30:40.849720 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40017
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:35563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:40017
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40389/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/testchainca.cert with env {}
W20260811 14:30:40.936635 30227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:40.936960 30227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:40.936987 30227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:40.938241 30227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:40.938292 30227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:40.938304 30227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:40.938318 30227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:40.938329 30227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:30:40.938342 30227 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:30:40.938354 30227 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40389/jwks.json
I20260811 14:30:40.940436 30227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:40017
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40389/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:40.940802 30227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:40.941058 30227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:40.946131 30236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:40.946090 30232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:40.946226 30227 server_base.cc:1034] running on GCE node
W20260811 14:30:40.946113 30234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:40.946614 30227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:40.946878 30227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:40.948045 30227 hybrid_clock.cc:648] HybridClock initialized: now 1786458640948018 us; error 36 us; skew 500 ppm
I20260811 14:30:40.949420 30227 webserver.cc:460] Webserver started at http://127.23.186.254:41741/ using document root <none> and password file <none>
I20260811 14:30:40.949682 30227 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:40.949761 30227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:40.949918 30227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:40.951072 30227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "c3cde8135678049e463f644e27f6c224"
server_key_iv: "c8206d000d1a7ef289a93926d9c146c0"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.951453 30227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "c3cde8135678049e463f644e27f6c224"
server_key_iv: "c8206d000d1a7ef289a93926d9c146c0"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.952982 30227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:40.953799 30243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:40.954000 30227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:40.954118 30227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10"
format_stamp: "Formatted at 2026-08-11 14:30:40 on dist-test-slave-cbsf"
server_key: "c3cde8135678049e463f644e27f6c224"
server_key_iv: "c8206d000d1a7ef289a93926d9c146c0"
server_key_version: "encryptionkey@0"
I20260811 14:30:40.954200 30227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:40.965976 30227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:40.966337 30227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:40.966496 30227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:40.970762 30227 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:40017
I20260811 14:30:40.970798 30307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:40017 every 8 connection(s)
I20260811 14:30:40.971203 30227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:40.972322 30308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:40.974886 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30227
I20260811 14:30:40.975028 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:40.974984 30308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10: Bootstrap starting.
I20260811 14:30:40.975204 24299 external_mini_cluster.cc:1428] Setting key e9e7c2397c522eb46c154e640ddce80e
I20260811 14:30:40.975885 30308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:40.976275 30308 log.cc:824] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:40.977222 30308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10: No bootstrap required, opened a new log
I20260811 14:30:40.979792 30308 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40017 } }
I20260811 14:30:40.980000 30308 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:40.980041 30308 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6ee37cbf9f64c1eb80a33fe681e1d10, State: Initialized, Role: FOLLOWER
I20260811 14:30:40.980214 30308 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40017 } }
I20260811 14:30:40.980324 30308 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:40.980369 30308 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:40.980417 30308 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:40.981429 30308 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40017 } }
I20260811 14:30:40.981559 30308 leader_election.cc:302] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6ee37cbf9f64c1eb80a33fe681e1d10; no voters:
I20260811 14:30:40.981757 30308 leader_election.cc:288] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:40.982055 30314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:40.982249 30314 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [term 1 LEADER]: Becoming Leader. State: Replica: f6ee37cbf9f64c1eb80a33fe681e1d10, State: Running, Role: LEADER
I20260811 14:30:40.982280 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:40.975826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40376 (local address 127.23.186.254:40017)
0811 14:30:40.976090 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:40.976101 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:40.976143 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:30:40.976817 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:40.976830 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:40.976930 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:40.977093 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:40.977320 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:40.978719 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:40.979549 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:40.980183 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:40.980327 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:40.981235 (+ 908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:40.981240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:40.981240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:40.981497 (+ 257us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:40.981548 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:40.981565 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:40.981651 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:40.981742 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 14:30:40.981958 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260811 14:30:40.982339 30308 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:40.982345 30314 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40017 } }
I20260811 14:30:40.982721 30315 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40017 } } }
I20260811 14:30:40.982815 30315 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:40.983060 30316 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6ee37cbf9f64c1eb80a33fe681e1d10. Latest consensus state: current_term: 1 leader_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6ee37cbf9f64c1eb80a33fe681e1d10" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40017 } } }
I20260811 14:30:40.983177 30316 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:40.983284 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:30:40.983490 30322 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:40.984079 30322 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:40.985795 30322 catalog_manager.cc:1347] Generated new cluster ID: 8e99869d772f4208b11894fe35a7fa08
I20260811 14:30:40.985867 30322 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:40.994889 30322 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:40.995716 30322 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:41.000517 30322 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6ee37cbf9f64c1eb80a33fe681e1d10: Generated new TSK 0
I20260811 14:30:41.000767 30322 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 14:30:41.032166 30345 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 14:30:41.032335 30345 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260811 14:30:41.033067 30344 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:41.016293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40017 (local address 127.0.0.1:40392)
0811 14:30:41.016520 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:41.016530 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:41.016594 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:41.016837 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:41.016838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:41.016840 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:30:41.016910 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.016912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.017889 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.017889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.018499 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.018500 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.018983 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.018984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.019031 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.019053 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:30:41.032445 (+ 13392us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40389/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 14:30:41.032579 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.186.254:40017: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40389/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260811 14:30:41.033560 30312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.016390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.23.186.254:40017)
0811 14:30:41.016455 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.016457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.016591 (+ 134us) server_negotiation.cc:407] Connection header received
0811 14:30:41.016685 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.016686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.016729 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.016809 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:30:41.016976 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.017810 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.018581 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.018891 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.018966 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.019166 (+ 200us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:30:41.032267 (+ 13101us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40389/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 14:30:41.033340 (+ 1073us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40389/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 14:30:41.033643 30312 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40392: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40389/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.23.186.254:40017: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40389/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 14:30:41.033924 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30227
2026-08-11T14:30:41Z chronyd exiting
I20260811 14:30:41.072211 24299 test_util.cc:175] -----------------------------------------------
I20260811 14:30:41.072301 24299 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786458616937656-24299-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 14:30:41.073176 24299 mini_oidc.cc:152] Starting JWKS server
I20260811 14:30:41.073912 24299 webserver.cc:460] Webserver started at https://127.0.0.1:43451/ using document root <none> and password file <none>
I20260811 14:30:41.073956 24299 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 14:30:41.074131 24299 webserver.cc:460] Webserver started at http://127.0.0.1:40429/ using document root <none> and password file <none>
2026-08-11T14:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:41Z Disabled control of system clock
I20260811 14:30:41.083676 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35759
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:35985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:35759
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43451/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/testchainca.cert with env {}
W20260811 14:30:41.166049 30351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:41.166285 30351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:41.166307 30351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:41.167452 30351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:41.167491 30351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:41.167505 30351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:41.167517 30351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:41.167529 30351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:30:41.167541 30351 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:30:41.167553 30351 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43451/jwks.json
I20260811 14:30:41.169096 30351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:35759
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43451/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:41.169318 30351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:41.169499 30351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:41.174707 30357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:41.174687 30356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:41.174793 30351 server_base.cc:1034] running on GCE node
W20260811 14:30:41.174687 30359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:41.175155 30351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:41.175505 30351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:41.176736 30351 hybrid_clock.cc:648] HybridClock initialized: now 1786458641176678 us; error 31 us; skew 500 ppm
I20260811 14:30:41.178184 30351 webserver.cc:460] Webserver started at http://127.23.186.254:42655/ using document root <none> and password file <none>
I20260811 14:30:41.178439 30351 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:41.178510 30351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:41.178663 30351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:41.179780 30351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "a029bb41a29e4c2382eb6da6ef7a3b87"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "b2c47a196bb2ea9b71bc0f2bfa8569a8"
server_key_iv: "1358ba13a91986ec2aabd27d08ecafd5"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.180171 30351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "a029bb41a29e4c2382eb6da6ef7a3b87"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "b2c47a196bb2ea9b71bc0f2bfa8569a8"
server_key_iv: "1358ba13a91986ec2aabd27d08ecafd5"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.181726 30351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:41.182515 30365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:41.182695 30351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:41.182808 30351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "a029bb41a29e4c2382eb6da6ef7a3b87"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "b2c47a196bb2ea9b71bc0f2bfa8569a8"
server_key_iv: "1358ba13a91986ec2aabd27d08ecafd5"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.182898 30351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:41.223040 30351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:41.223476 30351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:41.223658 30351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:41.228098 30351 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:35759
I20260811 14:30:41.228142 30430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:35759 every 8 connection(s)
I20260811 14:30:41.228533 30351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:41.229121 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30351
I20260811 14:30:41.229245 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:41.229440 24299 external_mini_cluster.cc:1428] Setting key 98ee50334198c0b15b962501d0af4382
I20260811 14:30:41.229522 30431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:41.232659 30431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87: Bootstrap starting.
I20260811 14:30:41.233355 30431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:41.233691 30431 log.cc:824] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:41.234517 30431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87: No bootstrap required, opened a new log
I20260811 14:30:41.235540 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.230268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.23.186.254:35759)
0811 14:30:41.230487 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.230495 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.230521 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:30:41.231000 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.231008 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.231083 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.231252 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:41.231497 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.232425 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.233339 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.233878 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.234017 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.234789 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:41.234794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:41.234794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:41.235014 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:41.235062 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:41.235078 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:41.235159 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:41.235220 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 14:30:41.235309 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":105,"threads_started":1}
I20260811 14:30:41.236721 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:30:41.237262 30431 raft_consensus.cc:348] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35759 } }
I20260811 14:30:41.237476 30431 raft_consensus.cc:374] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:41.237515 30431 raft_consensus.cc:729] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a029bb41a29e4c2382eb6da6ef7a3b87, State: Initialized, Role: FOLLOWER
I20260811 14:30:41.237640 30431 consensus_queue.cc:260] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35759 } }
I20260811 14:30:41.237725 30431 raft_consensus.cc:388] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:41.237783 30431 raft_consensus.cc:482] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:41.237833 30431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:41.238473 30431 raft_consensus.cc:504] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35759 } }
I20260811 14:30:41.238574 30431 leader_election.cc:302] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a029bb41a29e4c2382eb6da6ef7a3b87; no voters:
I20260811 14:30:41.238718 30431 leader_election.cc:288] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:41.238806 30438 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:41.239012 30438 raft_consensus.cc:686] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [term 1 LEADER]: Becoming Leader. State: Replica: a029bb41a29e4c2382eb6da6ef7a3b87, State: Running, Role: LEADER
I20260811 14:30:41.239027 30431 sys_catalog.cc:564] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:41.239130 30438 consensus_queue.cc:237] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35759 } }
I20260811 14:30:41.239542 30439 sys_catalog.cc:455] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35759 } } }
I20260811 14:30:41.239650 30439 sys_catalog.cc:458] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:41.239542 30440 sys_catalog.cc:455] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a029bb41a29e4c2382eb6da6ef7a3b87. Latest consensus state: current_term: 1 leader_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a029bb41a29e4c2382eb6da6ef7a3b87" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35759 } } }
I20260811 14:30:41.239809 30440 sys_catalog.cc:458] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:41.239957 30447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:41.240552 30447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:41.242241 30447 catalog_manager.cc:1347] Generated new cluster ID: 2ccfdc3d70514712ad66a404dc85bc27
I20260811 14:30:41.242296 30447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:41.249377 30447 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:41.249905 30447 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:41.257164 30447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a029bb41a29e4c2382eb6da6ef7a3b87: Generated new TSK 0
I20260811 14:30:41.257376 30447 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 14:30:41.278857 30464 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 14:30:41.278970 30464 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 14:30:41.279366 30435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.264368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48602 (local address 127.23.186.254:35759)
0811 14:30:41.264426 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.264428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.264518 (+ 90us) server_negotiation.cc:407] Connection header received
0811 14:30:41.264564 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.264565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.264579 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.264668 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:30:41.264845 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.265652 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.266441 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.266759 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.266833 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.267049 (+ 216us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:30:41.279100 (+ 12051us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 14:30:41.279287 (+ 187us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 14:30:41.279438 30435 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48602: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 14:30:41.279567 30463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:41.264289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35759 (local address 127.0.0.1:48602)
0811 14:30:41.264431 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:41.264442 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:41.264509 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:41.264666 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:41.264667 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:41.264668 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:30:41.264769 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.264771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.265740 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.265741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.266347 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.266348 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.266860 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.266862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.266912 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.266918 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:30:41.279269 (+ 12351us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 14:30:41.279397 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.186.254:35759: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43451/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":80,"threads_started":1}
I20260811 14:30:41.279911 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30351
2026-08-11T14:30:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (250 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 14:30:41.323196 24299 mini_oidc.cc:152] Starting JWKS server
I20260811 14:30:41.323943 24299 webserver.cc:460] Webserver started at https://127.0.0.1:34671/ using document root <none> and password file <none>
I20260811 14:30:41.324020 24299 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 14:30:41.324201 24299 webserver.cc:460] Webserver started at http://127.0.0.1:37325/ using document root <none> and password file <none>
2026-08-11T14:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:41Z Disabled control of system clock
I20260811 14:30:41.333846 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40817
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:34871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:40817
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34671/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 14:30:41.418787 30471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:41.419036 30471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:41.419061 30471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:41.420339 30471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:41.420389 30471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:41.420403 30471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:41.420446 30471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:41.420480 30471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:30:41.420497 30471 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:30:41.420517 30471 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34671/jwks.json
I20260811 14:30:41.422037 30471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:40817
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34671/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:41.422324 30471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:41.422644 30471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:41.427817 30478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:41.427824 30479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:41.427819 30481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:41.428174 30471 server_base.cc:1034] running on GCE node
I20260811 14:30:41.428471 30471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:41.428828 30471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:41.430011 30471 hybrid_clock.cc:648] HybridClock initialized: now 1786458641430000 us; error 42 us; skew 500 ppm
I20260811 14:30:41.431370 30471 webserver.cc:460] Webserver started at http://127.23.186.254:33845/ using document root <none> and password file <none>
I20260811 14:30:41.431643 30471 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:41.431713 30471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:41.431869 30471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:41.433072 30471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "65d43874111a45349aac380ad1f20039"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "7719c0e12e89d7c4332e51ce3b3134af"
server_key_iv: "f1085144b6e6c499de9e11b208dca0c1"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.433434 30471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "65d43874111a45349aac380ad1f20039"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "7719c0e12e89d7c4332e51ce3b3134af"
server_key_iv: "f1085144b6e6c499de9e11b208dca0c1"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.434755 30471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:41.435487 30488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:41.435678 30471 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:41.435786 30471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "65d43874111a45349aac380ad1f20039"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "7719c0e12e89d7c4332e51ce3b3134af"
server_key_iv: "f1085144b6e6c499de9e11b208dca0c1"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.435860 30471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:41.454650 30471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:41.455022 30471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:41.455188 30471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:41.459404 30471 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:40817
I20260811 14:30:41.459484 30550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:40817 every 8 connection(s)
I20260811 14:30:41.459841 30471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:41.460618 30551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:41.463680 30551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039: Bootstrap starting.
I20260811 14:30:41.464335 30551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:41.464638 30551 log.cc:824] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:41.465441 30551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039: No bootstrap required, opened a new log
I20260811 14:30:41.467077 30551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d43874111a45349aac380ad1f20039" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40817 } }
I20260811 14:30:41.467221 30551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:41.467245 30551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65d43874111a45349aac380ad1f20039, State: Initialized, Role: FOLLOWER
I20260811 14:30:41.467412 30551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d43874111a45349aac380ad1f20039" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40817 } }
I20260811 14:30:41.467480 30551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:41.467502 30551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:41.467557 30551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:41.468137 30551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d43874111a45349aac380ad1f20039" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40817 } }
I20260811 14:30:41.468252 30551 leader_election.cc:302] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65d43874111a45349aac380ad1f20039; no voters:
I20260811 14:30:41.468453 30551 leader_election.cc:288] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:41.468549 30554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:41.468752 30554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [term 1 LEADER]: Becoming Leader. State: Replica: 65d43874111a45349aac380ad1f20039, State: Running, Role: LEADER
I20260811 14:30:41.468794 30551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:41.468863 30554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d43874111a45349aac380ad1f20039" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40817 } }
I20260811 14:30:41.469084 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30471
I20260811 14:30:41.469182 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:41.469393 24299 external_mini_cluster.cc:1428] Setting key 5d33eacb04a3fdee19047be4111b1e85
I20260811 14:30:41.469435 30555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65d43874111a45349aac380ad1f20039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d43874111a45349aac380ad1f20039" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40817 } } }
I20260811 14:30:41.469521 30555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:41.469755 30557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65d43874111a45349aac380ad1f20039. Latest consensus state: current_term: 1 leader_uuid: "65d43874111a45349aac380ad1f20039" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65d43874111a45349aac380ad1f20039" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40817 } } }
I20260811 14:30:41.470018 30557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:41.470146 30566 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:41.470860 30566 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:41.472648 30566 catalog_manager.cc:1347] Generated new cluster ID: 9ef65504877d47a0b1e2d2ef57ff2b59
I20260811 14:30:41.472731 30566 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:41.476295 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.470091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.23.186.254:40817)
0811 14:30:41.470769 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.470778 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.470809 (+ 31us) server_negotiation.cc:407] Connection header received
0811 14:30:41.470934 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.470942 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.471023 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.471171 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:41.471526 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.472902 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.473985 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.474638 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.474782 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.475491 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:41.475496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:41.475496 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:41.475720 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:41.475768 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:41.475784 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:41.475857 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:41.475960 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 14:30:41.476055 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":101,"threads_started":1}
I20260811 14:30:41.477272 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:30:41.483530 30566 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:41.484256 30566 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:41.491285 30566 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65d43874111a45349aac380ad1f20039: Generated new TSK 0
I20260811 14:30:41.491508 30566 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:30:41.504840 30571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.501009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45804 (local address 127.23.186.254:40817)
0811 14:30:41.501073 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.501075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.501121 (+ 46us) server_negotiation.cc:407] Connection header received
0811 14:30:41.501209 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.501210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.501221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.501285 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:30:41.501419 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.501952 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.503101 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.503296 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.503348 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.503557 (+ 209us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:30:41.504550 (+ 993us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 14:30:41.504735 (+ 185us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45804: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 14:30:41.504900 30585 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:41.500879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40817 (local address 127.0.0.1:45804)
0811 14:30:41.501043 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:41.501063 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:41.501140 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:41.501286 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:41.501287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:41.501289 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:30:41.501348 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.501350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.502041 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.502042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.502997 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.503000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.503379 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.503380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.503444 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.503451 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:30:41.504660 (+ 1209us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 14:30:41.504797 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.186.254:40817: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34671/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":110,"threads_started":1}
I20260811 14:30:41.505180 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30471
E20260811 14:30:41.520984 30586 webserver.cc:533] Webserver: sslize failed:
E20260811 14:30:41.521090 30586 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T14:30:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T14:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:41Z Disabled control of system clock
I20260811 14:30:41.560575 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37251
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:35081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:37251
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/testchainca.cert with env {}
W20260811 14:30:41.645084 30590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:41.645344 30590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:41.645387 30590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:41.646682 30590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:41.646757 30590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:41.646781 30590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:41.646806 30590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:41.646833 30590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 14:30:41.646859 30590 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 14:30:41.646885 30590 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 14:30:41.648582 30590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:37251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:37251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:41.648907 30590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:41.649148 30590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:41.654326 30597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:41.654294 30598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:41.654318 30601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:41.654613 30590 server_base.cc:1034] running on GCE node
I20260811 14:30:41.654889 30590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:41.655220 30590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:41.656376 30590 hybrid_clock.cc:648] HybridClock initialized: now 1786458641656371 us; error 36 us; skew 500 ppm
I20260811 14:30:41.657799 30590 webserver.cc:460] Webserver started at http://127.23.186.254:34975/ using document root <none> and password file <none>
I20260811 14:30:41.658073 30590 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:41.658144 30590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:41.658296 30590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:41.659433 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "d6088028ab34403ca8546c4f9cedfb25"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "b8019fc3c105086d76219d062b437285"
server_key_iv: "ad00fb0be5f840b0017d8410a90abd5d"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.659798 30590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "d6088028ab34403ca8546c4f9cedfb25"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "b8019fc3c105086d76219d062b437285"
server_key_iv: "ad00fb0be5f840b0017d8410a90abd5d"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.661213 30590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:41.661967 30608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:41.662150 30590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:41.662259 30590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "d6088028ab34403ca8546c4f9cedfb25"
format_stamp: "Formatted at 2026-08-11 14:30:41 on dist-test-slave-cbsf"
server_key: "b8019fc3c105086d76219d062b437285"
server_key_iv: "ad00fb0be5f840b0017d8410a90abd5d"
server_key_version: "encryptionkey@0"
I20260811 14:30:41.662334 30590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:41.675953 30590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:41.676317 30590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:41.676492 30590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:41.680717 30590 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:37251
I20260811 14:30:41.680804 30676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:37251 every 8 connection(s)
I20260811 14:30:41.681164 30590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:41.682215 30677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:41.684715 30677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25: Bootstrap starting.
I20260811 14:30:41.685400 30677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:41.685504 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30590
I20260811 14:30:41.685621 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:41.685793 30677 log.cc:824] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:41.685822 24299 external_mini_cluster.cc:1428] Setting key 922bb5e9eb2f22475c0bb72c016958af
I20260811 14:30:41.686643 30677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25: No bootstrap required, opened a new log
I20260811 14:30:41.689230 30677 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6088028ab34403ca8546c4f9cedfb25" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37251 } }
I20260811 14:30:41.689414 30677 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:41.689445 30677 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6088028ab34403ca8546c4f9cedfb25, State: Initialized, Role: FOLLOWER
I20260811 14:30:41.689584 30677 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6088028ab34403ca8546c4f9cedfb25" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37251 } }
I20260811 14:30:41.689700 30677 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:41.689730 30677 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:41.689802 30677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:41.690554 30677 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6088028ab34403ca8546c4f9cedfb25" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37251 } }
I20260811 14:30:41.690697 30677 leader_election.cc:302] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6088028ab34403ca8546c4f9cedfb25; no voters:
I20260811 14:30:41.690889 30677 leader_election.cc:288] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:41.690939 30682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:41.691118 30682 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [term 1 LEADER]: Becoming Leader. State: Replica: d6088028ab34403ca8546c4f9cedfb25, State: Running, Role: LEADER
I20260811 14:30:41.691203 30677 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:41.691213 30682 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6088028ab34403ca8546c4f9cedfb25" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37251 } }
I20260811 14:30:41.691612 30682 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6088028ab34403ca8546c4f9cedfb25. Latest consensus state: current_term: 1 leader_uuid: "d6088028ab34403ca8546c4f9cedfb25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6088028ab34403ca8546c4f9cedfb25" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37251 } } }
I20260811 14:30:41.691623 30684 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6088028ab34403ca8546c4f9cedfb25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6088028ab34403ca8546c4f9cedfb25" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 37251 } } }
I20260811 14:30:41.691710 30682 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:41.691721 30684 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:41.691987 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.686365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53764 (local address 127.23.186.254:37251)
0811 14:30:41.686579 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.686586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.686611 (+ 25us) server_negotiation.cc:407] Connection header received
0811 14:30:41.687160 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.687169 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.687233 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.687355 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:41.687602 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.688480 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.689639 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.690192 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.690307 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.691149 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:41.691154 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:41.691154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:30:41.691371 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:41.691417 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:30:41.691433 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:41.691507 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:41.691630 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 14:30:41.691730 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":95,"threads_started":1}
I20260811 14:30:41.691992 30691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:41.692633 30691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:41.693500 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:30:41.695199 30691 catalog_manager.cc:1347] Generated new cluster ID: 1ca18611c23f4de6981f3af17f6d085f
I20260811 14:30:41.695251 30691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:41.702948 30691 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:41.703467 30691 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:41.707588 30691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6088028ab34403ca8546c4f9cedfb25: Generated new TSK 0
I20260811 14:30:41.707782 30691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:30:41.780117 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:41.718694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.23.186.254:37251)
0811 14:30:41.718773 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:41.718776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:41.718885 (+ 109us) server_negotiation.cc:407] Connection header received
0811 14:30:41.719012 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:41.719015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:41.719033 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:41.719122 (+ 89us) server_negotiation.cc:226] Negotiated authn=JWT
0811 14:30:41.719275 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.719823 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.720651 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.720922 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.721002 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.721216 (+ 214us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 14:30:41.779795 (+ 58579us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 14:30:41.780026 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53776: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 14:30:41.780212 30681 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53776: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 14:30:41.780210 30709 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:41.718619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:37251 (local address 127.0.0.1:53776)
0811 14:30:41.718811 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:41.718826 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:41.718905 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:41.719123 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:41.719124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:41.719126 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0811 14:30:41.719195 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.719198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.719932 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.719933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.720554 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:41.720556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:41.721010 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:41.721011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:41.721075 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:41.721088 (+ 13us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 14:30:41.779960 (+ 58872us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 14:30:41.780097 (+ 137us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.23.186.254:37251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":112,"threads_started":1}
I20260811 14:30:41.780659 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30590
2026-08-11T14:30:41Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (249 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (94 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (103 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:42 dist-test-slave-cbsf krb5kdc[30724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:42 dist-test-slave-cbsf krb5kdc[30724](info): set up 2 sockets
Aug 11 14:30:42 dist-test-slave-cbsf krb5kdc[30724](info): commencing operation
krb5kdc: starting...
W20260811 14:30:43.126478 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:43.253859 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:46633
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:43.338881 30739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:43.339210 30739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:43.339238 30739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:43.340494 30739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:43.340544 30739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:43.340636 30739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:43.340713 30739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:43.340736 30739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:43.342303 30739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:43.342608 30739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:43.342871 30739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:43.347985 30746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:43.347990 30745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:43.348073 30739 server_base.cc:1034] running on GCE node
W20260811 14:30:43.348006 30748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:43.348486 30739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:43.348877 30739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:43.350061 30739 hybrid_clock.cc:648] HybridClock initialized: now 1786458643350050 us; error 51 us; skew 500 ppm
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.355778 30739 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:43.356316 30739 webserver.cc:460] Webserver started at http://127.23.186.254:45727/ using document root <none> and password file <none>
I20260811 14:30:43.356593 30739 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:43.356666 30739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:43.356882 30739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:43.358019 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "f8343562bc4d4213978312f1ed3a45f2"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "5ae496217711b59ca64f67e7db84d4b9"
server_key_iv: "20f8c3fa258faec625127684b6b2ecc2"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.358389 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "f8343562bc4d4213978312f1ed3a45f2"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "5ae496217711b59ca64f67e7db84d4b9"
server_key_iv: "20f8c3fa258faec625127684b6b2ecc2"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.359927 30739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:43.360901 30760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.361117 30739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:43.361250 30739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "f8343562bc4d4213978312f1ed3a45f2"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "5ae496217711b59ca64f67e7db84d4b9"
server_key_iv: "20f8c3fa258faec625127684b6b2ecc2"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.361346 30739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:43.380225 30739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:43.382452 30739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:43.382664 30739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:43.387620 30739 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:46633
I20260811 14:30:43.387653 30820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:46633 every 8 connection(s)
I20260811 14:30:43.388104 30739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:43.389153 30821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:43.389967 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30739
I20260811 14:30:43.390120 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:43.390348 24299 external_mini_cluster.cc:1428] Setting key 70cebc0b5d3b9fb68c654dcdf1aefe93
I20260811 14:30:43.392319 30821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2: Bootstrap starting.
I20260811 14:30:43.393013 30821 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:43.393349 30821 log.cc:824] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:43.394209 30821 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2: No bootstrap required, opened a new log
I20260811 14:30:43.396409 30821 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8343562bc4d4213978312f1ed3a45f2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46633 } }
I20260811 14:30:43.396622 30821 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:43.396669 30821 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8343562bc4d4213978312f1ed3a45f2, State: Initialized, Role: FOLLOWER
I20260811 14:30:43.396855 30821 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8343562bc4d4213978312f1ed3a45f2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46633 } }
I20260811 14:30:43.396945 30821 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:43.396970 30821 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:43.397037 30821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:43.397640 30821 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8343562bc4d4213978312f1ed3a45f2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46633 } }
I20260811 14:30:43.397730 30821 leader_election.cc:302] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8343562bc4d4213978312f1ed3a45f2; no voters:
I20260811 14:30:43.397926 30821 leader_election.cc:288] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:43.398000 30829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:43.398216 30821 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:43.398317 30829 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [term 1 LEADER]: Becoming Leader. State: Replica: f8343562bc4d4213978312f1ed3a45f2, State: Running, Role: LEADER
I20260811 14:30:43.398437 30829 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8343562bc4d4213978312f1ed3a45f2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46633 } }
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:43.398886 30830 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8343562bc4d4213978312f1ed3a45f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8343562bc4d4213978312f1ed3a45f2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46633 } } }
I20260811 14:30:43.398989 30830 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:43.399168 30830 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8343562bc4d4213978312f1ed3a45f2. Latest consensus state: current_term: 1 leader_uuid: "f8343562bc4d4213978312f1ed3a45f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8343562bc4d4213978312f1ed3a45f2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46633 } } }
I20260811 14:30:43.399220 30841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:43.399238 30830 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:43.399747 30841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:43.401633 30841 catalog_manager.cc:1347] Generated new cluster ID: f40d952da1b045a2b480c4fdde6e1558
I20260811 14:30:43.401706 30841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:43.404502 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.391027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.23.186.254:46633)
0811 14:30:43.391266 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.391277 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.391316 (+ 39us) server_negotiation.cc:407] Connection header received
0811 14:30:43.392007 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.392018 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.392097 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.392255 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.393128 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.394517 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.395358 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.396072 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.396198 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.399583 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.399588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.399588 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.399604 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.402622 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.403094 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.403098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.403098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.403217 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.403453 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.403455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.403455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.403805 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.403986 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.404107 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.404259 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":118,"threads_started":1}
I20260811 14:30:43.434211 30841 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:43.434834 30841 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:43.443881 30841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8343562bc4d4213978312f1ed3a45f2: Generated new TSK 0
I20260811 14:30:43.444123 30841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:43.474022 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46633
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:43.558982 30858 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:43.559207 30858 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:43.559231 30858 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:43.560346 30858 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:43.560386 30858 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:43.560412 30858 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:43.561894 30858 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30858
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:43.562104 30858 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:43.562302 30858 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:43.567422 30866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:43.567541 30858 server_base.cc:1034] running on GCE node
W20260811 14:30:43.567392 30864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:43.567408 30863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:43.567875 30858 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:43.568145 30858 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:43.569289 30858 hybrid_clock.cc:648] HybridClock initialized: now 1786458643569273 us; error 27 us; skew 500 ppm
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.571918 30858 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:43.572368 30858 webserver.cc:460] Webserver started at http://127.23.186.193:44273/ using document root <none> and password file <none>
I20260811 14:30:43.572607 30858 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:43.572678 30858 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:43.572863 30858 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:43.573939 30858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "5f2bea725d4c416793490d920ad5616d"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "f035e82adb9e82e2c952c0830a02785a"
server_key_iv: "ce910b5b12f44e769633566e452c42fe"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.574330 30858 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "5f2bea725d4c416793490d920ad5616d"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "f035e82adb9e82e2c952c0830a02785a"
server_key_iv: "ce910b5b12f44e769633566e452c42fe"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.575697 30858 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:43.576417 30875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.576603 30858 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:43.576735 30858 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "5f2bea725d4c416793490d920ad5616d"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "f035e82adb9e82e2c952c0830a02785a"
server_key_iv: "ce910b5b12f44e769633566e452c42fe"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.576843 30858 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:43.592177 30858 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:43.594661 30858 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:43.594842 30858 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:43.595104 30858 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:43.595449 30858 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:43.595515 30858 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.595561 30858 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:43.595587 30858 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.602032 30858 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:36493
I20260811 14:30:43.602099 31009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:36493 every 8 connection(s)
I20260811 14:30:43.602423 30858 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:43.610180 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 30858
I20260811 14:30:43.610307 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:43.610515 24299 external_mini_cluster.cc:1428] Setting key da1fc200f1b4a8c8e378eaa920285270
I20260811 14:30:43.612032 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.603197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:52967 (local address 127.23.186.254:46633)
0811 14:30:43.603243 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.603245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.603472 (+ 227us) server_negotiation.cc:407] Connection header received
0811 14:30:43.604400 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.604401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.604412 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.604464 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.605113 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.605631 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.606607 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.606818 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.606878 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.609157 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.609158 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.609159 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.609166 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.610511 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.611004 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.611005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.611005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.611072 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.611344 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.611345 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.611345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.611474 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.611554 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.611845 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.611943 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:30:43.612280 31012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.603178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:46633 (local address 127.23.186.193:52967)
0811 14:30:43.603347 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:43.603383 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:43.604301 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:43.604543 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:43.604546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:43.604835 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:43.605035 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:43.605040 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.605717 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.605717 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:43.606521 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:43.606523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.606905 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.606906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:43.607010 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.607594 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:43.607603 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:43.609055 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:43.610625 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:43.610632 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:43.610635 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:43.610931 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:43.611148 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:43.611149 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:43.611149 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:43.611275 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:43.611549 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:43.611556 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:43.611703 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 14:30:43.611808 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 14:30:43.611978 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":99,"threads_started":1}
I20260811 14:30:43.612795 31010 heartbeater.cc:347] Connected to a master server at 127.23.186.254:46633
I20260811 14:30:43.612893 31010 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:43.613152 31010 heartbeater.cc:511] Master 127.23.186.254:46633 requested a full tablet report, sending...
I20260811 14:30:43.613759 30778 ts_manager.cc:194] Registered new tserver with Master: 5f2bea725d4c416793490d920ad5616d (127.23.186.193:36493)
I20260811 14:30:43.614578 30778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:52967
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:43.637876 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46633
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:43.721573 31015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:43.721797 31015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:43.721822 31015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:43.723016 31015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:43.723067 31015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:43.723094 31015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:43.724617 31015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:43.724902 31015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:43.725165 31015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:43.730235 31028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:43.730260 31022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:43.730235 31024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:43.730474 31015 server_base.cc:1034] running on GCE node
I20260811 14:30:43.730698 31015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:43.730949 31015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:43.732111 31015 hybrid_clock.cc:648] HybridClock initialized: now 1786458643732096 us; error 28 us; skew 500 ppm
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.734822 31015 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:43.735258 31015 webserver.cc:460] Webserver started at http://127.23.186.194:42777/ using document root <none> and password file <none>
I20260811 14:30:43.735484 31015 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:43.735561 31015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:43.735713 31015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:43.736816 31015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "0312424f70b94a57ad1f61dddc95825d"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "c561a5c7d307aa8f5cd8deb5de99a507"
server_key_iv: "347a1495f4f0116f6435b30a21988877"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.737195 31015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "0312424f70b94a57ad1f61dddc95825d"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "c561a5c7d307aa8f5cd8deb5de99a507"
server_key_iv: "347a1495f4f0116f6435b30a21988877"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.738510 31015 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:43.739234 31035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.739403 31015 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:43.739512 31015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "0312424f70b94a57ad1f61dddc95825d"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "c561a5c7d307aa8f5cd8deb5de99a507"
server_key_iv: "347a1495f4f0116f6435b30a21988877"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.739601 31015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:43.750766 31015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:43.753194 31015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:43.753365 31015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:43.753620 31015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:43.753940 31015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:43.753996 31015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.754042 31015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:43.754066 31015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.760218 31015 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:39095
I20260811 14:30:43.760269 31173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:39095 every 8 connection(s)
I20260811 14:30:43.760605 31015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:43.763643 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31015
I20260811 14:30:43.763778 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:43.763970 24299 external_mini_cluster.cc:1428] Setting key ef4b8fedf92d80a576f2f49ff4b38f2d
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:43.771037 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.761326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:45889 (local address 127.23.186.254:46633)
0811 14:30:43.761399 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.761401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.761615 (+ 214us) server_negotiation.cc:407] Connection header received
0811 14:30:43.762612 (+ 997us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.762614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.762627 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.762688 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.763315 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.763924 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.765256 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.765540 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.765625 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.768131 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.768132 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.768133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.768141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.769605 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.770039 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.770040 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.770041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.770101 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.770366 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.770367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.770367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.770488 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.770562 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.770850 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.770945 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:30:43.771230 31177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.761309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:46633 (local address 127.23.186.194:45889)
0811 14:30:43.761492 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:43.761527 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:43.762492 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:43.762753 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:43.762764 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:43.763017 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:43.763211 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:43.763215 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.764019 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.764020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:43.765163 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:43.765166 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.765638 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.765639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:43.765804 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.766389 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:43.766401 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:43.768029 (+ 1628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:43.769703 (+ 1674us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:43.769710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:43.769713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:43.769946 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:43.770177 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:43.770177 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:43.770178 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:43.770291 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:43.770565 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:43.770572 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:43.770713 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 14:30:43.770825 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:30:43.770991 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260811 14:30:43.771667 31174 heartbeater.cc:347] Connected to a master server at 127.23.186.254:46633
I20260811 14:30:43.771762 31174 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:43.771971 31174 heartbeater.cc:511] Master 127.23.186.254:46633 requested a full tablet report, sending...
I20260811 14:30:43.772401 30778 ts_manager.cc:194] Registered new tserver with Master: 0312424f70b94a57ad1f61dddc95825d (127.23.186.194:39095)
I20260811 14:30:43.772994 30778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:45889
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:43.789783 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46633
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41887
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:43.872572 31180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:43.872802 31180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:43.872828 31180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:43.873996 31180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:43.874040 31180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:43.874074 31180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:43.875656 31180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41887
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46633
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:43.875919 31180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:43.876165 31180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:43.881307 31187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:43.881378 31190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:43.881337 31188 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:43.881811 31180 server_base.cc:1034] running on GCE node
I20260811 14:30:43.882023 31180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:43.882251 31180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:43.883440 31180 hybrid_clock.cc:648] HybridClock initialized: now 1786458643883419 us; error 37 us; skew 500 ppm
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.886201 31180 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:43.886607 31180 webserver.cc:460] Webserver started at http://127.23.186.195:46457/ using document root <none> and password file <none>
I20260811 14:30:43.886855 31180 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:43.886904 31180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:43.887012 31180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:43.888046 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "2bff364202b049a6ab66cc4bdb2578c7"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "390dd1359ca6a49f6b1ed4984725ac25"
server_key_iv: "46c878725f451a25aadbb06a4459302e"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.888495 31180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "2bff364202b049a6ab66cc4bdb2578c7"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "390dd1359ca6a49f6b1ed4984725ac25"
server_key_iv: "46c878725f451a25aadbb06a4459302e"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.889942 31180 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:43.890637 31197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.890959 31180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:43.891042 31180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "2bff364202b049a6ab66cc4bdb2578c7"
format_stamp: "Formatted at 2026-08-11 14:30:43 on dist-test-slave-cbsf"
server_key: "390dd1359ca6a49f6b1ed4984725ac25"
server_key_iv: "46c878725f451a25aadbb06a4459302e"
server_key_version: "encryptionkey@0"
I20260811 14:30:43.891155 31180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:43.906654 31180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:43.908617 31180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:43.908826 31180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:43.909068 31180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:43.909390 31180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:43.909425 31180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.909467 31180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:43.909508 31180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:43.915477 31180 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:34157
I20260811 14:30:43.915567 31338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:34157 every 8 connection(s)
I20260811 14:30:43.915894 31180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:43.925303 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31180
I20260811 14:30:43.925395 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:43.925575 24299 external_mini_cluster.cc:1428] Setting key 1327fb1fb68c8eb54134feb26d0f860f
I20260811 14:30:43.925936 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.916635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:59547 (local address 127.23.186.254:46633)
0811 14:30:43.916687 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.916713 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.916899 (+ 186us) server_negotiation.cc:407] Connection header received
0811 14:30:43.917915 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.917917 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.917931 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.917991 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.918657 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.919184 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.920222 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.920427 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.920510 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.922847 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.922849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.922849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.922856 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.924319 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.924803 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.924808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.924808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.924864 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.925145 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.925147 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.925147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.925312 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.925388 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.925756 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.925845 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:30:43.926178 31342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.916590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:46633 (local address 127.23.186.195:59547)
0811 14:30:43.916762 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:43.916812 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:43.917796 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:43.918072 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:43.918077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:43.918357 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:43.918556 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:43.918560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.919295 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.919296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:43.920123 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:43.920126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.920519 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.920520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:43.920666 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.921294 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:43.921303 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:43.922738 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:43.924410 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:43.924417 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:43.924419 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:43.924656 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:43.924926 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:43.924926 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:43.924927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:43.925036 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:43.925406 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:43.925438 (+ 32us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:43.925574 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 14:30:43.925714 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 14:30:43.925887 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260811 14:30:43.926680 31340 heartbeater.cc:347] Connected to a master server at 127.23.186.254:46633
I20260811 14:30:43.926959 31340 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:43.927165 31340 heartbeater.cc:511] Master 127.23.186.254:46633 requested a full tablet report, sending...
I20260811 14:30:43.927543 30778 ts_manager.cc:194] Registered new tserver with Master: 2bff364202b049a6ab66cc4bdb2578c7 (127.23.186.195:34157)
I20260811 14:30:43.927956 30778 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:59547
I20260811 14:30:43.937923 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/test-admin.keytab.
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.953670 24299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 14:30:43.955426 31348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:43.954577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34160 (local address 127.23.186.193:36493)
0811 14:30:43.954780 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.954785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.954796 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:43.954823 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.954825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.954838 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.954906 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.955361 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34160: BlockingRecv error: recv got EOF from 127.0.0.1:34160 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":117,"threads_started":1}
W20260811 14:30:43.955588 24299 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc' of type FILE
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.956686 24299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.957755 24299 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:43.966141 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.958323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.23.186.193:36493)
0811 14:30:43.958381 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.958383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.958392 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:43.958435 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.958436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.958444 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.958492 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.958902 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.959643 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.960407 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.960800 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.960856 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.963146 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.963156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.963156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.963163 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.964901 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.965243 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.965247 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.965247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.965304 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.965557 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.965558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.965558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.965701 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.965809 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.965880 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.965998 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
W20260811 14:30:43.968005 31348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:43.967471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34174 (local address 127.23.186.193:36493)
0811 14:30:43.967512 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.967514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.967567 (+ 53us) server_negotiation.cc:407] Connection header received
0811 14:30:43.967619 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.967620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.967629 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.967671 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.967937 (+ 266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34174: BlockingRecv error: recv got EOF from 127.0.0.1:34174 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 14:30:43.968068 24299 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc' of type FILE
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.969120 24299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.970001 24299 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:43.976960 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.970546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34176 (local address 127.23.186.193:36493)
0811 14:30:43.970584 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.970586 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.970646 (+ 60us) server_negotiation.cc:407] Connection header received
0811 14:30:43.970686 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.970687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.970697 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.970743 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.971098 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.971580 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.972252 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.972440 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.972488 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.974527 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.974528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.974528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.974535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.975711 (+ 1176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.976060 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.976061 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.976062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.976119 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.976449 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.976450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.976450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.976556 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.976634 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.976812 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.976889 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:30:43.978636 31348 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:43.978024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.23.186.193:36493)
0811 14:30:43.978086 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.978088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.978142 (+ 54us) server_negotiation.cc:407] Connection header received
0811 14:30:43.978214 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.978215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.978224 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.978289 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.978586 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34192: BlockingRecv error: recv got EOF from 127.0.0.1:34192 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 14:30:43.978740 24299 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786458616937656-24299-0/krb5kdc/krb5cc' of type FILE
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.979808 24299 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:43.980823 24299 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 14:30:43 dist-test-slave-cbsf krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:30:43.987661 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:43.981319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34206 (local address 127.23.186.193:36493)
0811 14:30:43.981379 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:43.981380 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:43.981422 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:30:43.981480 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:43.981481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:43.981488 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:43.981535 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:43.981866 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.982332 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.982985 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:43.983171 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:43.983244 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:43.985309 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:43.985310 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:43.985310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:43.985316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:43.986496 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.986790 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.986791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.986791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.986838 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:43.987104 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:43.987105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:43.987105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:43.987212 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:43.987322 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:43.987440 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 14:30:43.987500 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:30:43.988350 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30858
I20260811 14:30:43.996194 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31015
I20260811 14:30:44.003822 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31180
I20260811 14:30:44.011722 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 30739
2026-08-11T14:30:44Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2036 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:44 dist-test-slave-cbsf krb5kdc[31364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:44 dist-test-slave-cbsf krb5kdc[31364](info): set up 2 sockets
Aug 11 14:30:44 dist-test-slave-cbsf krb5kdc[31364](info): commencing operation
krb5kdc: starting...
W20260811 14:30:45.143791 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:45Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab.
Entry for principal oryx/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab.
I20260811 14:30:45.272387 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:42483
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab
--principal=oryx/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:42483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:45.356313 31379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:45.356613 31379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:45.356655 31379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:45.357920 31379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:45.358007 31379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:45.358032 31379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:45.358060 31379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:45.358093 31379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:45.359645 31379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:42483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab
--principal=oryx/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:42483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:45.359983 31379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:45.360244 31379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:45.365314 31385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.365483 31379 server_base.cc:1034] running on GCE node
W20260811 14:30:45.365536 31384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:45.365315 31387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.365904 31379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:45.366236 31379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:45.367420 31379 hybrid_clock.cc:648] HybridClock initialized: now 1786458645367394 us; error 58 us; skew 500 ppm
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:45.370182 31379 init.cc:377] Logged in from keytab as oryx/127.23.186.254@KRBTEST.COM (short username oryx)
I20260811 14:30:45.370729 31379 webserver.cc:460] Webserver started at http://127.23.186.254:38825/ using document root <none> and password file <none>
I20260811 14:30:45.370975 31379 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:45.371047 31379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:45.371207 31379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:45.372330 31379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "760dc8b205a441b0bf16670e8dd716a7"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "e40863a0d4a985049fb82ff3b3732acc"
server_key_iv: "f7d7bd9f5a233956fa218a938becddb6"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.372789 31379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "760dc8b205a441b0bf16670e8dd716a7"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "e40863a0d4a985049fb82ff3b3732acc"
server_key_iv: "f7d7bd9f5a233956fa218a938becddb6"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.374233 31379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:45.375140 31395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.375345 31379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:45.375454 31379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "760dc8b205a441b0bf16670e8dd716a7"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "e40863a0d4a985049fb82ff3b3732acc"
server_key_iv: "f7d7bd9f5a233956fa218a938becddb6"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.375553 31379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:45.396391 31379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:45.398988 31379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:45.399174 31379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:45.403627 31379 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:42483
I20260811 14:30:45.403684 31462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:42483 every 8 connection(s)
I20260811 14:30:45.404088 31379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:45.405143 31463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:45.407528 31463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Bootstrap starting.
I20260811 14:30:45.408254 31463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:45.408504 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31379
I20260811 14:30:45.408571 31463 log.cc:824] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:45.408627 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:45.408850 24299 external_mini_cluster.cc:1428] Setting key ce22498afe83af2eb59205d9995900e6
I20260811 14:30:45.409583 31463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: No bootstrap required, opened a new log
I20260811 14:30:45.412348 31463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } }
I20260811 14:30:45.412529 31463 raft_consensus.cc:374] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:45.412585 31463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760dc8b205a441b0bf16670e8dd716a7, State: Initialized, Role: FOLLOWER
I20260811 14:30:45.412781 31463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } }
I20260811 14:30:45.412879 31463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:45.412904 31463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:45.412972 31463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:45.413662 31463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } }
I20260811 14:30:45.413775 31463 leader_election.cc:302] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 760dc8b205a441b0bf16670e8dd716a7; no voters:
I20260811 14:30:45.413944 31463 leader_election.cc:288] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:45.414019 31468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:45.414218 31468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 LEADER]: Becoming Leader. State: Replica: 760dc8b205a441b0bf16670e8dd716a7, State: Running, Role: LEADER
I20260811 14:30:45.414201 31463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:45.414374 31468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } }
I20260811 14:30:45.415071 31471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 760dc8b205a441b0bf16670e8dd716a7. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } }
I20260811 14:30:45.415159 31471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:45.415247 31469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } }
I20260811 14:30:45.415306 31469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:45.416119 31478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:45.417421 31478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:45.419214 31478 catalog_manager.cc:1347] Generated new cluster ID: a1e47cbad435406387ae424fea8d7e30
I20260811 14:30:45.419269 31478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:45.420959 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.409416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54864 (local address 127.23.186.254:42483)
0811 14:30:45.409646 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:45.409655 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:45.409692 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:30:45.410219 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:45.410228 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:45.410293 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:45.410443 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:45.411034 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.412512 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.413468 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.414125 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.414569 (+ 444us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.417175 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:45.417179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:45.417179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:45.417193 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:45.419252 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.419669 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.419672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.419672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.419766 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.420013 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.420014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.420014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.420301 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:45.420451 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:45.420591 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 14:30:45.420749 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":122,"threads_started":1}
I20260811 14:30:45.457544 31478 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:45.458143 31478 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:45.468499 31478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Generated new TSK 0
I20260811 14:30:45.468750 31478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab.
Entry for principal oryx/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab.
I20260811 14:30:45.497602 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42483
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:45.582795 31491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:45.583102 31491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:45.583134 31491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:45.584363 31491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:45.584414 31491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:45.584441 31491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:45.585990 31491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:45.586278 31491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:45.586537 31491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:45.591797 31497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.591814 31491 server_base.cc:1034] running on GCE node
W20260811 14:30:45.591796 31503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:45.591828 31501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.592310 31491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:45.592607 31491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:45.593783 31491 hybrid_clock.cc:648] HybridClock initialized: now 1786458645593782 us; error 54 us; skew 500 ppm
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:45.596522 31491 init.cc:377] Logged in from keytab as oryx/127.23.186.193@KRBTEST.COM (short username oryx)
I20260811 14:30:45.597038 31491 webserver.cc:460] Webserver started at http://127.23.186.193:45049/ using document root <none> and password file <none>
I20260811 14:30:45.597254 31491 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:45.597298 31491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:45.597401 31491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:45.598457 31491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "65c9aa5e5d004db183da0827fc6d75c5"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "8f8c5c700c3b7d54f6e43842485a1720"
server_key_iv: "eed6f01a01f3afa32384933149856ad7"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.598778 31491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "65c9aa5e5d004db183da0827fc6d75c5"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "8f8c5c700c3b7d54f6e43842485a1720"
server_key_iv: "eed6f01a01f3afa32384933149856ad7"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.600069 31491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:45.600831 31515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.601007 31491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:45.601099 31491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "65c9aa5e5d004db183da0827fc6d75c5"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "8f8c5c700c3b7d54f6e43842485a1720"
server_key_iv: "eed6f01a01f3afa32384933149856ad7"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.601157 31491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:45.625566 31491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:45.627533 31491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:45.627664 31491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:45.627874 31491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:45.628227 31491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:45.628264 31491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.628288 31491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:45.628340 31491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.634285 31491 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:36245
I20260811 14:30:45.634398 31654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:36245 every 8 connection(s)
I20260811 14:30:45.634768 31491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:45.643985 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31491
I20260811 14:30:45.644074 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:45.644272 24299 external_mini_cluster.cc:1428] Setting key a5a6765a2611577edcce126862703d0a
I20260811 14:30:45.644383 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.635599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:41085 (local address 127.23.186.254:42483)
0811 14:30:45.635649 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:45.635651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:45.635842 (+ 191us) server_negotiation.cc:407] Connection header received
0811 14:30:45.636376 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:45.636377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:45.636392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:45.636453 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:45.637459 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.637984 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.639011 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.639216 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.639276 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.641546 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:45.641548 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:45.641548 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:45.641556 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:45.642780 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.643282 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.643284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.643284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.643354 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.643630 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.643631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.643632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.643767 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:45.643844 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:45.644184 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 14:30:45.644288 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:30:45.644619 31658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.635573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:41085)
0811 14:30:45.635736 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:45.635772 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:45.636261 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:45.636508 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:45.636511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:45.637169 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:45.637374 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:45.637379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.638083 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.638083 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:45.638921 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:45.638923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.639299 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.639300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:45.639405 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.639899 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:45.639907 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:45.641436 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:45.642902 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:45.642909 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:45.642911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:45.643154 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:45.643431 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:45.643432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:45.643433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:45.643557 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:45.643846 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:45.643854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:45.644002 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 14:30:45.644157 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 14:30:45.644337 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260811 14:30:45.645128 31655 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42483
I20260811 14:30:45.645248 31655 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:45.645437 31655 heartbeater.cc:511] Master 127.23.186.254:42483 requested a full tablet report, sending...
I20260811 14:30:45.646059 31417 ts_manager.cc:194] Registered new tserver with Master: 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:45.646981 31417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:41085
WARNING: no policy specified for oryx/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab.
Entry for principal oryx/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab.
I20260811 14:30:45.671404 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42483
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:45.755452 31662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:45.755685 31662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:45.755712 31662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:45.757042 31662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:45.757154 31662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:45.757208 31662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:45.758854 31662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:45.759158 31662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:45.759428 31662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:45.764833 31667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:45.764824 31670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:45.764858 31668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.765242 31662 server_base.cc:1034] running on GCE node
I20260811 14:30:45.765488 31662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:45.765707 31662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:45.766851 31662 hybrid_clock.cc:648] HybridClock initialized: now 1786458645766841 us; error 27 us; skew 500 ppm
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:45.769515 31662 init.cc:377] Logged in from keytab as oryx/127.23.186.194@KRBTEST.COM (short username oryx)
I20260811 14:30:45.769982 31662 webserver.cc:460] Webserver started at http://127.23.186.194:34645/ using document root <none> and password file <none>
I20260811 14:30:45.770253 31662 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:45.770327 31662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:45.770483 31662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:45.771618 31662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "07b4360466d842848a1fb5033a4ccf22"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "30a40e78ecbb0c691fc949eb569576e0"
server_key_iv: "fe879d9da0e67af5b74471ae1006e457"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.771989 31662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "07b4360466d842848a1fb5033a4ccf22"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "30a40e78ecbb0c691fc949eb569576e0"
server_key_iv: "fe879d9da0e67af5b74471ae1006e457"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.773459 31662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:45.774235 31680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.774427 31662 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:45.774540 31662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "07b4360466d842848a1fb5033a4ccf22"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "30a40e78ecbb0c691fc949eb569576e0"
server_key_iv: "fe879d9da0e67af5b74471ae1006e457"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.774619 31662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:45.787227 31662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:45.789695 31662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:45.789875 31662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:45.790133 31662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:45.790458 31662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:45.790514 31662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.790553 31662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:45.790601 31662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.796610 31662 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33035
I20260811 14:30:45.796654 31812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33035 every 8 connection(s)
I20260811 14:30:45.797079 31662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:45.806870 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.797980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:42311 (local address 127.23.186.254:42483)
0811 14:30:45.798034 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:45.798037 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:45.798284 (+ 247us) server_negotiation.cc:407] Connection header received
0811 14:30:45.798768 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:45.798769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:45.798780 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:45.798836 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:45.799772 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.800290 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.801371 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.801572 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.801633 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.803873 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:45.803874 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:45.803874 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:45.803883 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:45.805433 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.805872 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.805873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.805873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.805942 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.806198 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.806199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.806199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.806326 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:45.806421 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:45.806687 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 14:30:45.806771 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:45.807094 31817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.797966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:42311)
0811 14:30:45.798154 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:45.798192 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:45.798682 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:45.798888 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:45.798892 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:45.799478 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:45.799688 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:45.799693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.800376 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.800377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:45.801289 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:45.801292 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.801653 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.801654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:45.801763 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.802280 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:45.802290 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:45.803759 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:45.805527 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:45.805536 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:45.805539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:45.805790 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:45.806013 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:45.806014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:45.806014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:45.806128 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:45.806396 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:45.806404 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:45.806549 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 14:30:45.806664 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 14:30:45.806835 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
I20260811 14:30:45.807284 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31662
I20260811 14:30:45.807376 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:45.807596 24299 external_mini_cluster.cc:1428] Setting key 1a8e2452c691264335e363c17cbf5cca
I20260811 14:30:45.807648 31813 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42483
I20260811 14:30:45.807756 31813 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:45.807972 31813 heartbeater.cc:511] Master 127.23.186.254:42483 requested a full tablet report, sending...
I20260811 14:30:45.808401 31417 ts_manager.cc:194] Registered new tserver with Master: 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035)
I20260811 14:30:45.808910 31417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:42311
WARNING: no policy specified for oryx/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab.
Entry for principal oryx/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab.
I20260811 14:30:45.835700 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42483
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:45.921176 31821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:45.921435 31821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:45.921478 31821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:45.922724 31821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:45.922801 31821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:45.922847 31821 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:45.924429 31821 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42483
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31821
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:45.924768 31821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:45.925024 31821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:45.930377 31830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.930459 31821 server_base.cc:1034] running on GCE node
W20260811 14:30:45.930377 31827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:45.930408 31826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:45.930811 31821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:45.931162 31821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:45.932339 31821 hybrid_clock.cc:648] HybridClock initialized: now 1786458645932323 us; error 45 us; skew 500 ppm
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:45.935261 31821 init.cc:377] Logged in from keytab as oryx/127.23.186.195@KRBTEST.COM (short username oryx)
I20260811 14:30:45.935789 31821 webserver.cc:460] Webserver started at http://127.23.186.195:34985/ using document root <none> and password file <none>
I20260811 14:30:45.936046 31821 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:45.936120 31821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:45.936275 31821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:45.937491 31821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "c599349c89f047c2975dbea16464dca1"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "78853d13fbbeed03471548dab7c70f63"
server_key_iv: "0f6972d017e257b1664231356c9e6472"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.937889 31821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "c599349c89f047c2975dbea16464dca1"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "78853d13fbbeed03471548dab7c70f63"
server_key_iv: "0f6972d017e257b1664231356c9e6472"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.939313 31821 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:30:45.940140 31838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.940325 31821 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:45.940433 31821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "c599349c89f047c2975dbea16464dca1"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "78853d13fbbeed03471548dab7c70f63"
server_key_iv: "0f6972d017e257b1664231356c9e6472"
server_key_version: "encryptionkey@0"
I20260811 14:30:45.940527 31821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:45.955994 31821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:45.958635 31821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:45.958925 31821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:45.959290 31821 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:45.959692 31821 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:45.959756 31821 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.959805 31821 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:45.959851 31821 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:45.967140 31821 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:43197
I20260811 14:30:45.967177 31973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:43197 every 8 connection(s)
I20260811 14:30:45.967585 31821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:45.972069 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31821
I20260811 14:30:45.972150 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:45.972362 24299 external_mini_cluster.cc:1428] Setting key 52af1739d194c7296d3f62f09ded2549
Aug 11 14:30:45 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:45.978139 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.968511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:60787 (local address 127.23.186.254:42483)
0811 14:30:45.968581 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:45.968583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:45.968847 (+ 264us) server_negotiation.cc:407] Connection header received
0811 14:30:45.969395 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:45.969397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:45.969409 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:45.969488 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:45.970570 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.971097 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.972279 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.972566 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.972676 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.975258 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:45.975260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:45.975260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:45.975268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:45.976529 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.977051 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.977053 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.977053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.977134 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:45.977448 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:45.977449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:45.977449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:45.977570 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:45.977624 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:45.977994 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 14:30:45.978068 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:45.978376 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:45.968476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:60787)
0811 14:30:45.968671 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:45.968740 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:45.969272 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:45.969543 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:45.969547 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:45.970183 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:45.970455 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:45.970461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.971205 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.971206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:45.972104 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:45.972107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:45.972713 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:45.972715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:45.972863 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:45.973511 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:45.973520 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:45.975133 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:45.976612 (+ 1479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:45.976619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:45.976622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:45.976970 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:45.977237 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:45.977238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:45.977238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:45.977360 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:45.977697 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:45.977704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:45.977856 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 14:30:45.977946 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 14:30:45.978101 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":110,"threads_started":1}
I20260811 14:30:45.978952 31974 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42483
I20260811 14:30:45.979063 31974 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:45.979231 31974 heartbeater.cc:511] Master 127.23.186.254:42483 requested a full tablet report, sending...
I20260811 14:30:45.979699 31417 ts_manager.cc:194] Registered new tserver with Master: c599349c89f047c2975dbea16464dca1 (127.23.186.195:43197)
I20260811 14:30:45.980046 31417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:60787
I20260811 14:30:45.984098 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.186.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.253.keytab.
Entry for principal oryx/127.23.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.253.keytab.
I20260811 14:30:46.012228 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:34957
--webserver_interface=127.23.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.253.keytab
--principal=oryx/127.23.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:42483,127.23.186.253:34957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:46.099088 31979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:46.099373 31979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:46.099413 31979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:46.100611 31979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:46.100692 31979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:46.100740 31979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:46.100765 31979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:46.100795 31979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:46.102341 31979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:42483,127.23.186.253:34957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.253.keytab
--principal=oryx/127.23.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:34957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
W20260811 14:30:46.102499 31979 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.23.186.254:42483,127.23.186.253:34957'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 14:30:46.102677 31979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:46.102916 31979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:46.107973 31986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.107966 31989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.108006 31987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.108454 31979 server_base.cc:1034] running on GCE node
I20260811 14:30:46.108785 31979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:46.109159 31979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:46.110360 31979 hybrid_clock.cc:648] HybridClock initialized: now 1786458646110352 us; error 34 us; skew 500 ppm
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:46.113166 31979 init.cc:377] Logged in from keytab as oryx/127.23.186.253@KRBTEST.COM (short username oryx)
I20260811 14:30:46.113682 31979 webserver.cc:460] Webserver started at http://127.23.186.253:33313/ using document root <none> and password file <none>
I20260811 14:30:46.113967 31979 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.114058 31979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.114216 31979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:46.115660 31979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/instance:
uuid: "15528f761e2e46dca9dfd2c4be1d7808"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "b5b176a8ea400c3b7bef8435a709eb4b"
server_key_iv: "a6711973dba135e9b54bd5c41e22bd93"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.116084 31979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal/instance:
uuid: "15528f761e2e46dca9dfd2c4be1d7808"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "b5b176a8ea400c3b7bef8435a709eb4b"
server_key_iv: "a6711973dba135e9b54bd5c41e22bd93"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.117749 31979 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:30:46.118609 32000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.118793 31979 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:46.118919 31979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
uuid: "15528f761e2e46dca9dfd2c4be1d7808"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "b5b176a8ea400c3b7bef8435a709eb4b"
server_key_iv: "a6711973dba135e9b54bd5c41e22bd93"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.119006 31979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.136358 31979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:46.138571 31979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:46.138770 31979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:46.143227 31979 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.253:34957
I20260811 14:30:46.143276 32062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.253:34957 every 8 connection(s)
I20260811 14:30:46.143838 31979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
I20260811 14:30:46.145272 32064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:46.146857 32064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:30:46.149072 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 31979
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.253@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
I20260811 14:30:46.157656 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.147717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54876 (local address 127.23.186.254:42483)
0811 14:30:46.147815 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.147817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.148002 (+ 185us) server_negotiation.cc:407] Connection header received
0811 14:30:46.148462 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.148463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.148474 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.148556 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.149311 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.149993 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.151249 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.151446 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.151506 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.154348 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.154351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.154351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.154361 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.155937 (+ 1576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.156378 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.156380 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.156381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.156446 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.156742 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.156743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.156744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.156917 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.156998 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.157399 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.157537 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 14:30:46.157850 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.147666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:54876)
0811 14:30:46.147851 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.147896 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.148376 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.148608 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.148611 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.148952 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.149209 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.149219 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.150143 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.150144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.151158 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.151161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.151530 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.151531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.151674 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.152293 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.152302 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.154245 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.156029 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.156037 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.156040 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.156291 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.156519 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.156521 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.156521 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.156648 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.157001 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.157008 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.157232 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.157353 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.157537 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":102,"threads_started":1}
I20260811 14:30:46.158533 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.149604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35864 (local address 127.23.186.253:34957)
0811 14:30:46.149927 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.149931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.149947 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:30:46.149966 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.149967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.149983 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.150042 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.150756 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.151715 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.152441 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.152903 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.152981 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.155360 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.155369 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.155370 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.155377 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.157027 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.157455 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.157459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.157459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.157528 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.157758 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.157759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.157759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.158083 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.158275 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.158324 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.158414 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":103,"threads_started":1}
I20260811 14:30:46.158660 32064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.253@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
I20260811 14:30:46.165064 32067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.158966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35872)
0811 14:30:46.159011 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.159021 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.159066 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.159185 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.159186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.159319 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.159356 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.159357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.159974 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.159975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.160482 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.160483 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.160877 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.160878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.160923 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.161301 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.161307 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.162504 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.164157 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.164158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.164158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.164329 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.164506 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.164506 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.164507 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.164569 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.164818 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.164819 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.164857 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.164907 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.164991 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 14:30:46.165196 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.159035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35872 (local address 127.23.186.253:34957)
0811 14:30:46.159070 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.159072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.159097 (+ 25us) server_negotiation.cc:407] Connection header received
0811 14:30:46.159118 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.159118 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.159128 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.159176 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.159420 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.159892 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.160553 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.160819 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.160867 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.162588 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.162589 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.162589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.162596 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.164064 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.164397 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.164398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.164398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.164448 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.164608 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.164608 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.164608 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.164764 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.164827 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.164931 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.165043 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":17}
I20260811 14:30:46.166636 32064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Bootstrap starting.
I20260811 14:30:46.167514 32064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:46.167831 32064 log.cc:824] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:46.168625 32064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: No bootstrap required, opened a new log
I20260811 14:30:46.170118 32064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } }
I20260811 14:30:46.170275 32064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:46.170328 32064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15528f761e2e46dca9dfd2c4be1d7808, State: Initialized, Role: FOLLOWER
I20260811 14:30:46.170518 32064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } }
I20260811 14:30:46.170861 32064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:46.170821 32071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } } }
I20260811 14:30:46.170934 32071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:30:46.172608 32085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:30:46.172667 32085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:30:46.174706 31979 master_runner.cc:428] Detected that this master 15528f761e2e46dca9dfd2c4be1d7808 is joining an existing cluster
I20260811 14:30:46.174805 31979 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.186.253:34957
I20260811 14:30:46.184096 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.176080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35880 (local address 127.23.186.253:34957)
0811 14:30:46.176148 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.176151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.176505 (+ 354us) server_negotiation.cc:407] Connection header received
0811 14:30:46.176555 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.176556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.176565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.176593 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.177311 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.177977 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.179195 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.179410 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.179441 (+ 31us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.180854 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.180855 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.180855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.180862 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.182792 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.183182 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.183183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.183184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.183232 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.183417 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.183418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.183418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.183544 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.183588 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.183769 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.183996 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":157,"server-negotiator.queue_time_us":41}
I20260811 14:30:46.184500 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.175928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35880)
0811 14:30:46.176415 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.176427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.176495 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.176669 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.176669 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.177076 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.177233 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.177235 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.178054 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.178054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.179100 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.179102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.179505 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.179506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.179540 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.180111 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.180116 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.180752 (+ 636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.182876 (+ 2124us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.182877 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.182877 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.183104 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.183299 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.183300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.183300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.183352 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.183627 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.183628 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.183664 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.183745 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.183939 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":33,"thread_start_us":87,"threads_started":1}
I20260811 14:30:46.185380 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.176012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54882 (local address 127.23.186.254:42483)
0811 14:30:46.176132 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.176134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.176536 (+ 402us) server_negotiation.cc:407] Connection header received
0811 14:30:46.176565 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.176566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.176578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.176660 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.177308 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.178157 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.179194 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.179507 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.179591 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.181876 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.181878 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.181878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.181886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.183508 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.184035 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.184036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.184037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.184103 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.184617 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.184618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.184619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.184924 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.184975 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.185251 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.185319 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 14:30:46.185413 32092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.176271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:54882)
0811 14:30:46.176415 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.176427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.176495 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.176670 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.176671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.177075 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.177212 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.177216 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.178264 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.178265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.179100 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.179102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.179600 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.179601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.179683 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.180294 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.180298 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.181799 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.183608 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.183609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.183610 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.183931 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.184441 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.184442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.184442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.184525 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.185101 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.185101 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.185146 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.185208 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.185307 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":7,"thread_start_us":83,"threads_started":1}
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.254@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
I20260811 14:30:46.194731 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.187630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35884 (local address 127.23.186.253:34957)
0811 14:30:46.187674 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.187676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.187812 (+ 136us) server_negotiation.cc:407] Connection header received
0811 14:30:46.187918 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.187919 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.187928 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.187973 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.188411 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.189001 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.189787 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.189986 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.190075 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.192017 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.192018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.192019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.192025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.193493 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.193895 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.193897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.193897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.193957 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.194173 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.194173 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.194174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.194280 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.194349 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.194545 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.194632 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:46.194749 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.187552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35884)
0811 14:30:46.187700 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.187749 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.187839 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.187976 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.187980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.188268 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.188316 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.188318 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.189084 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.189085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.189683 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.189685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.190077 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.190078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.190161 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.190597 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.190601 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.191916 (+ 1315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.193597 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.193605 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.193608 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.193816 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.194024 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.194025 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.194025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.194101 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.194353 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.194361 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.194414 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.194506 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.194638 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260811 14:30:46.195353 31417 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.186.253:34957: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808"
member_type: NON_VOTER
last_known_addr {
host: "127.23.186.253"
port: 34957
}
attrs {
promote: true
}
}
dest_uuid: "760dc8b205a441b0bf16670e8dd716a7"
cas_config_opid_index: -1
I20260811 14:30:46.195741 31417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } }
I20260811 14:30:46.196262 32099 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.196632 32034 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:46.197548 32099 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.186.253:34957
I20260811 14:30:46.197707 32034 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Refusing update from remote peer 760dc8b205a441b0bf16670e8dd716a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 14:30:46.197885 31979 master.cc:537] Master@127.23.186.253:34957 shutting down...
I20260811 14:30:46.199036 32099 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 14:30:46.199983 31396 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 14:30:46.200317 32098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.200413 32098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
W20260811 14:30:46.200390 31396 consensus_peers.cc:487] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 -> Peer 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253:34957): Couldn't send request to peer 15528f761e2e46dca9dfd2c4be1d7808. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 14:30:46.201068 31979 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:46.201166 31979 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:46.201222 31979 tablet_replica.cc:330] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: stopping tablet replica
I20260811 14:30:46.214653 31979 master.cc:552] Master@127.23.186.253:34957 shutdown complete.
I20260811 14:30:46.214727 31979 master_runner.cc:305] Clearing existing system tablet
I20260811 14:30:46.214870 31979 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.214922 31979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.215754 31979 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:46.216326 32103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.216467 31979 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.216527 31979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.216555 31979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
uuid: "15528f761e2e46dca9dfd2c4be1d7808"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "b5b176a8ea400c3b7bef8435a709eb4b"
server_key_iv: "a6711973dba135e9b54bd5c41e22bd93"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.217365 31979 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:46.218888 31979 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 14:30:46.218953 31979 log.cc:1196] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 14:30:46.219295 31979 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Deleting consensus metadata
I20260811 14:30:46.219444 31979 master_runner.cc:315] Copying system tablet from 127.23.186.254:42483
I20260811 14:30:46.220163 31979 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: tablet copy: Beginning tablet copy session from remote peer at address 127.23.186.254:42483
I20260811 14:30:46.226629 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.220734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54884 (local address 127.23.186.254:42483)
0811 14:30:46.220783 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.220786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.220844 (+ 58us) server_negotiation.cc:407] Connection header received
0811 14:30:46.220923 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.220924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.220935 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.220987 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.221332 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.221857 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.222647 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.222854 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.222935 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.223998 (+ 1063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.223999 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.223999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.224008 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.225404 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.225778 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.225780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.225780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.225844 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.226068 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.226069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.226070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.226194 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.226273 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.226446 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.226528 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:30:46.226650 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.220611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:54884)
0811 14:30:46.220770 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.220783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.220863 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.220991 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.220991 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.221181 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.221235 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.221237 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.221960 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.221961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.222536 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.222537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.222936 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.222937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.223009 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.223568 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.223573 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.223895 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.225501 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.225502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.225503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.225699 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.225921 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.225922 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.225923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.225989 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.226263 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.226264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.226315 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.226403 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.226529 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":136,"threads_started":1}
I20260811 14:30:46.227090 31446 tablet_copy_service.cc:140] P 760dc8b205a441b0bf16670e8dd716a7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 15528f761e2e46dca9dfd2c4be1d7808 ({username='oryx', principal='oryx/127.23.186.253@KRBTEST.COM'} at 127.0.0.1:54884)
I20260811 14:30:46.227216 31446 tablet_copy_service.cc:161] P 760dc8b205a441b0bf16670e8dd716a7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 15528f761e2e46dca9dfd2c4be1d7808 at {username='oryx', principal='oryx/127.23.186.253@KRBTEST.COM'} at 127.0.0.1:54884: session id = 15528f761e2e46dca9dfd2c4be1d7808-00000000000000000000000000000000
I20260811 14:30:46.228004 31446 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 14:30:46.228777 31979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 14:30:46.229895 31979 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: tablet copy: Starting download of 0 data blocks...
I20260811 14:30:46.230077 31979 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: tablet copy: Starting download of 1 WAL segments...
I20260811 14:30:46.231132 31979 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 14:30:46.232024 31446 tablet_copy_service.cc:342] P 760dc8b205a441b0bf16670e8dd716a7: Request end of tablet copy session 15528f761e2e46dca9dfd2c4be1d7808-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.186.253@KRBTEST.COM'} at 127.0.0.1:54884
I20260811 14:30:46.232095 31446 tablet_copy_service.cc:433] P 760dc8b205a441b0bf16670e8dd716a7: ending tablet copy session 15528f761e2e46dca9dfd2c4be1d7808-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 15528f761e2e46dca9dfd2c4be1d7808
I20260811 14:30:46.234565 31979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:46.235718 32111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.235755 32112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.235945 31979 server_base.cc:1034] running on GCE node
W20260811 14:30:46.237434 32114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.237581 31979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:46.237758 31979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:46.238886 31979 hybrid_clock.cc:648] HybridClock initialized: now 1786458646238880 us; error 25 us; skew 500 ppm
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:46.240216 31979 init.cc:377] Logged in from keytab as oryx/127.23.186.253@KRBTEST.COM (short username oryx)
I20260811 14:30:46.240520 31979 webserver.cc:460] Webserver started at http://127.23.186.253:43495/ using document root <none> and password file <none>
I20260811 14:30:46.240682 31979 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.240751 31979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.241406 31979 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:46.241953 32122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.242079 31979 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:46.242125 31979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
uuid: "15528f761e2e46dca9dfd2c4be1d7808"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "b5b176a8ea400c3b7bef8435a709eb4b"
server_key_iv: "a6711973dba135e9b54bd5c41e22bd93"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.242182 31979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.252561 31979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:46.254213 31979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:46.257476 31979 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.253:34957
I20260811 14:30:46.257792 32184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.253:34957 every 8 connection(s)
I20260811 14:30:46.257899 31979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
I20260811 14:30:46.258858 32185 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:30:46.259431 32185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Bootstrap starting.
I20260811 14:30:46.262641 32185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:30:46.262913 32185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: Bootstrap complete.
I20260811 14:30:46.263159 32185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } }
I20260811 14:30:46.263258 32185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 15528f761e2e46dca9dfd2c4be1d7808, State: Initialized, Role: LEARNER
I20260811 14:30:46.263653 32185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } }
I20260811 14:30:46.263878 32185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:46.266135 32202 catalog_manager.cc:1259] Loaded cluster ID: a1e47cbad435406387ae424fea8d7e30
I20260811 14:30:46.266642 32202 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: loading cluster ID for follower catalog manager: success
I20260811 14:30:46.267756 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.260154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35898 (local address 127.23.186.253:34957)
0811 14:30:46.260339 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.260340 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.260347 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:46.260367 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.260367 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.260376 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.260450 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.261216 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.261889 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.262622 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.262930 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.263011 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.264449 (+ 1438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.264451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.264451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.264459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.266259 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.266685 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.266686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.266687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.266761 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.266986 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.266987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.266988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.267158 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.267271 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.267566 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.267666 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260811 14:30:46.268574 32202 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: acquiring CA information for follower catalog manager: success
I20260811 14:30:46.269012 32202 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:30:46.272869 32190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.273008 32190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: LEARNER
WARNING: no policy specified for oryx/127.23.186.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.252.keytab.
Entry for principal oryx/127.23.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.252.keytab.
I20260811 14:30:46.296187 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:43991
--webserver_interface=127.23.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.252.keytab
--principal=oryx/127.23.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:46.383200 32205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:46.383469 32205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:46.383512 32205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:46.384761 32205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:46.384842 32205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:46.384867 32205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:46.384896 32205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:46.384927 32205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:46.386538 32205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.252.keytab
--principal=oryx/127.23.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:43991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:46.386847 32205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:46.387102 32205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:46.392412 32212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.392403 32213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.392448 32216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.392678 32205 server_base.cc:1034] running on GCE node
I20260811 14:30:46.393055 32205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:46.393369 32205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:46.394541 32205 hybrid_clock.cc:648] HybridClock initialized: now 1786458646394520 us; error 51 us; skew 500 ppm
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:46.397176 32205 init.cc:377] Logged in from keytab as oryx/127.23.186.252@KRBTEST.COM (short username oryx)
I20260811 14:30:46.397634 32205 webserver.cc:460] Webserver started at http://127.23.186.252:37171/ using document root <none> and password file <none>
I20260811 14:30:46.397866 32205 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.397913 32205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.398084 32205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:46.399173 32205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/instance:
uuid: "e431a103e3264c75ae234353c0158cc6"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "4b8d7e5b7ff7a377f85be86ce276f708"
server_key_iv: "32d73b758acfd346ebad9d8742b9a65f"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.399487 32205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal/instance:
uuid: "e431a103e3264c75ae234353c0158cc6"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "4b8d7e5b7ff7a377f85be86ce276f708"
server_key_iv: "32d73b758acfd346ebad9d8742b9a65f"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.400846 32205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:46.401585 32226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.401774 32205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:46.401850 32205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
uuid: "e431a103e3264c75ae234353c0158cc6"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "4b8d7e5b7ff7a377f85be86ce276f708"
server_key_iv: "32d73b758acfd346ebad9d8742b9a65f"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.401897 32205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.419675 32205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:46.422194 32205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:46.422317 32205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:46.427117 32205 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.252:43991
I20260811 14:30:46.427191 32288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.252:43991 every 8 connection(s)
I20260811 14:30:46.427544 32205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
I20260811 14:30:46.428581 32289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:46.430246 32289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:30:46.432757 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 32205
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.252@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.186.252@KRBTEST.COM
I20260811 14:30:46.440263 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.431142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54900 (local address 127.23.186.254:42483)
0811 14:30:46.431245 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.431248 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.431451 (+ 203us) server_negotiation.cc:407] Connection header received
0811 14:30:46.432025 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.432027 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.432041 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.432116 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.432800 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.433514 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.434550 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.434746 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.434837 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.437215 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.437217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.437218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.437228 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.438791 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.439252 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.439253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.439253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.439319 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.439579 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.439580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.439580 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.439717 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.439793 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.440074 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.440160 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 14:30:46.440479 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.430989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:54900)
0811 14:30:46.431315 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.431354 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.431891 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.432196 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.432200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.432467 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.432663 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.432667 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.433614 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.433615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.434439 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.434441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.434882 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.434884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.435005 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.435572 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.435581 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.437105 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.438885 (+ 1780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.438893 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.438896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.439163 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.439391 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.439392 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.439392 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.439505 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.439798 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.439805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.439940 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.440052 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.440238 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":132,"threads_started":1}
I20260811 14:30:46.441145 32289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:30:46.442404 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.433312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55320 (local address 127.23.186.252:43991)
0811 14:30:46.433496 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.433500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.433515 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:30:46.433535 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.433537 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.433560 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.433650 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.434244 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.435002 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.435714 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.436010 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.436089 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.438950 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.438957 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.438957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.438964 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.440989 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.441467 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.441470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.441471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.441561 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.441826 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.441826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.441827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.442010 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.442188 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.442275 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.442336 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":96,"threads_started":1}
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.252@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
I20260811 14:30:46.448134 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.441418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35912)
0811 14:30:46.441474 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.441487 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.441524 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.441748 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.441749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.441901 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.441964 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.441965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.442905 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.442906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.443560 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.443562 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.443900 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.443901 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.443954 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.444356 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.444360 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.445681 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.447041 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.447042 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.447043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.447214 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.447456 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.447457 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.447458 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.447522 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.447797 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.447798 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.447845 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.447935 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.448039 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 14:30:46.448134 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.441553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35912 (local address 127.23.186.253:34957)
0811 14:30:46.441627 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.441629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.441637 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:46.441665 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.441666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.441681 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.441746 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.442043 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.442825 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.443636 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.443836 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.443891 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.445798 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.445800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.445801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.445816 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.446941 (+ 1125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.447299 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.447300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.447300 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.447359 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.447586 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.447587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.447587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.447711 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.447805 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.447961 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.448045 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:30:46.448678 32289 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.252@KRBTEST.COM for oryx/127.23.186.252@KRBTEST.COM
I20260811 14:30:46.454743 32291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.449009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.0.0.1:55330)
0811 14:30:46.449069 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.449080 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.449125 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.449238 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.449239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.449355 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.449388 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.449389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.449989 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.449989 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.450449 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.450450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.450775 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.450776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.450813 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.451173 (+ 360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.451177 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.452296 (+ 1119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.453891 (+ 1595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.453891 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.453892 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.454055 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.454236 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.454237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.454237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.454288 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.454505 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.454506 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.454541 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.454593 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.454672 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:30:46.454800 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.449061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.23.186.252:43991)
0811 14:30:46.449114 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.449116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.449125 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:46.449172 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.449173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.449182 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.449226 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.449432 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.449927 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.450517 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.450715 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.450766 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.452373 (+ 1607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.452375 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.452375 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.452381 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.453812 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.454121 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.454122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.454122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.454171 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.454340 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.454341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.454341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.454453 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.454503 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.454609 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.454716 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":34}
I20260811 14:30:46.456454 32289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Bootstrap starting.
I20260811 14:30:46.457443 32289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:46.457836 32289 log.cc:824] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:46.458654 32289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: No bootstrap required, opened a new log
I20260811 14:30:46.460201 32289 raft_consensus.cc:348] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } }
I20260811 14:30:46.460332 32289 raft_consensus.cc:374] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:46.460355 32289 raft_consensus.cc:729] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e431a103e3264c75ae234353c0158cc6, State: Initialized, Role: FOLLOWER
I20260811 14:30:46.460527 32289 consensus_queue.cc:260] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } }
I20260811 14:30:46.460852 32295 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } } }
I20260811 14:30:46.460919 32289 sys_catalog.cc:564] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:46.460970 32295 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:30:46.462821 32308 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:30:46.462880 32308 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:30:46.465426 32205 master_runner.cc:428] Detected that this master e431a103e3264c75ae234353c0158cc6 is joining an existing cluster
I20260811 14:30:46.465489 32205 master_runner.cc:432] Initiating AddMaster RPC to add 127.23.186.252:43991
I20260811 14:30:46.474464 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.466692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35922 (local address 127.23.186.253:34957)
0811 14:30:46.466740 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.466742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.467112 (+ 370us) server_negotiation.cc:407] Connection header received
0811 14:30:46.467206 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.467208 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.467220 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.467294 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.467609 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.468477 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.469622 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.469923 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.469965 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.471311 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.471313 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.471313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.471322 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.473169 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.473562 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.473563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.473564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.473624 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.473850 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.473851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.473852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.474000 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.474084 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.474267 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.474351 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:30:46.474471 32320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.466589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35922)
0811 14:30:46.467030 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.467052 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.467140 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.467295 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.467296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.467474 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.467535 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.467537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.468559 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.468560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.469531 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.469533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.470072 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.470073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.470134 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.470695 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.470700 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.471190 (+ 490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.473240 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.473241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.473242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.473454 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.473705 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.473706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.473707 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.473782 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.474084 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.474086 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.474139 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.474222 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.474337 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":94,"threads_started":1}
I20260811 14:30:46.475991 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.466781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54902 (local address 127.23.186.254:42483)
0811 14:30:46.466856 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.466858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.468889 (+ 2031us) server_negotiation.cc:407] Connection header received
0811 14:30:46.468948 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.468950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.468965 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.469026 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.469528 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.470245 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.471387 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.471710 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.471755 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.472906 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.472907 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.472908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.472916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.474694 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.475196 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.475198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.475198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.475262 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.475446 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.475447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.475447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.475573 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.475648 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.475813 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.475892 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:30:46.476011 32316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.466603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:54902)
0811 14:30:46.468803 (+ 2200us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.468815 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.468890 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.469077 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.469078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.469333 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.469429 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.469431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.470336 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.470337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.471294 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.471297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.471837 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.471838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.471912 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.472302 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.472307 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.472796 (+ 489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.474897 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.474898 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.474898 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.475085 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.475321 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.475321 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.475321 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.475378 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.475652 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.475653 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.475696 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.475783 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.475894 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2177,"spinlock_wait_cycles":12160,"thread_start_us":90,"threads_started":1}
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458645, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.254@KRBTEST.COM for oryx/127.23.186.252@KRBTEST.COM
I20260811 14:30:46.492802 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.478155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.0.0.1:55354)
0811 14:30:46.478196 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.478208 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.478246 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.478561 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.478563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.478747 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.478796 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.478797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.479539 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.479541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.480376 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.480378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.480851 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.480852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.480926 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.481307 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.481312 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.482717 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.491075 (+ 8358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.491077 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.491078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.491358 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.491613 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.491614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.491615 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.491697 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.492321 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.492323 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.492430 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.492514 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.492657 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 14:30:46.492908 32292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.467066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55338 (local address 127.23.186.252:43991)
0811 14:30:46.467129 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.467131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.469038 (+ 1907us) server_negotiation.cc:407] Connection header received
0811 14:30:46.469116 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.469117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.469125 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.469151 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.469618 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.470287 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.471442 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.471740 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.471785 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.490010 (+ 18225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.490012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.490013 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.490023 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.491704 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.492070 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.492071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.492071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.492157 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.492386 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.492387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.492387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.492505 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.492560 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.492736 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.492815 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:30:46.493084 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.478219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55354 (local address 127.23.186.252:43991)
0811 14:30:46.478389 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.478390 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.478396 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:30:46.478427 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.478427 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.478436 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.478518 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.478874 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.479458 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.480470 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.480779 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.480836 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.482837 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.482839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.482839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.482846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.490887 (+ 8041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.491456 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.491457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.491458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.491529 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.492072 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.492073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.492073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.492251 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.492318 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.492563 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.492655 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":115,"threads_started":1}
I20260811 14:30:46.493767 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31491
I20260811 14:30:46.493799 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.466863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.0.0.1:55338)
0811 14:30:46.468987 (+ 2124us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.468996 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.469056 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.469203 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.469204 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.469471 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.469532 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.469534 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.470383 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.470383 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.471333 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.471337 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.471861 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.471861 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.471922 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.489355 (+ 17433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.489362 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.489888 (+ 526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.491776 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.491777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.491778 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.491985 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.492236 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.492238 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.492238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.492321 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.492581 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.492581 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.492614 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.493564 (+ 950us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.493691 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2103,"mutex_wait_us":90,"thread_start_us":111,"threads_started":1}
I20260811 14:30:46.493767 31417 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.23.186.252:43991: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e431a103e3264c75ae234353c0158cc6"
member_type: NON_VOTER
last_known_addr {
host: "127.23.186.252"
port: 43991
}
attrs {
promote: true
}
}
dest_uuid: "760dc8b205a441b0bf16670e8dd716a7"
cas_config_opid_index: 5
I20260811 14:30:46.494047 31417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } }
I20260811 14:30:46.496284 32098 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e431a103e3264c75ae234353c0158cc6 (127.23.186.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.497440 32257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:46.498282 32257 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Refusing update from remote peer 760dc8b205a441b0bf16670e8dd716a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 14:30:46.499125 32098 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.23.186.252:43991
I20260811 14:30:46.499220 32098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:46.499498 32098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.499585 32098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:46.499841 32205 master.cc:537] Master@127.23.186.252:43991 shutting down...
I20260811 14:30:46.501187 32257 raft_consensus.cc:858] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } }
W20260811 14:30:46.501260 32257 raft_consensus.cc:1565] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } }
I20260811 14:30:46.503008 32257 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.504212 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.504420 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 14:30:46.504494 32295 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 760dc8b205a441b0bf16670e8dd716a7. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.504652 32295 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 14:30:46.504633 32332 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } }
I20260811 14:30:46.504752 32332 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 14:30:46.505280 32099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.505395 32099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
W20260811 14:30:46.506052 31398 consensus_peers.cc:487] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 -> Peer e431a103e3264c75ae234353c0158cc6 (127.23.186.252:43991): Couldn't send request to peer e431a103e3264c75ae234353c0158cc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 14:30:46.506170 32325 mvcc.cc:205] Tried to move back new op lower bound from 7317334611836817408 to 7317334611617636352. Current Snapshot: MvccSnapshot[applied={T|T < 7317334611836817408 or (T in {7317334611836817408})}]
I20260811 14:30:46.507943 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.496863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35936)
0811 14:30:46.504752 (+ 7889us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.504767 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.504863 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.505109 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.505111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.505112 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:46.505170 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.505172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.506118 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.506119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.506978 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.506982 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.507633 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.507635 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.507705 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.507744 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.507812 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.507864 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7862}
I20260811 14:30:46.507943 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.496986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35936 (local address 127.23.186.253:34957)
0811 14:30:46.504618 (+ 7632us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.504621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.504851 (+ 230us) server_negotiation.cc:407] Connection header received
0811 14:30:46.504943 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.504944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.504970 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.505040 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:46.505262 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.506012 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.507103 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.507518 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.507633 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.507714 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.507820 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.507865 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7593}
I20260811 14:30:46.508296 32154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 LEARNER]: Refusing update from remote peer 760dc8b205a441b0bf16670e8dd716a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 14:30:46.508570 32323 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 14:30:46.509438 32154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER e431a103e3264c75ae234353c0158cc6 (127.23.186.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.510953 32334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.511070 32334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:30:46.511247 32190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 760dc8b205a441b0bf16670e8dd716a7. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: NON_VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: true } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.511299 32190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:30:46.511602 32323 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: attempting to promote NON_VOTER 15528f761e2e46dca9dfd2c4be1d7808 to VOTER
I20260811 14:30:46.511788 32323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } }
I20260811 14:30:46.512498 32154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 LEARNER]: Refusing update from remote peer 760dc8b205a441b0bf16670e8dd716a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260811 14:30:46.512575 31398 consensus_peers.cc:487] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 -> Peer e431a103e3264c75ae234353c0158cc6 (127.23.186.252:43991): Couldn't send request to peer e431a103e3264c75ae234353c0158cc6. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 14:30:46.512737 32098 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 14:30:46.513607 32154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.514451 32190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.514530 32190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:30:46.513454 32323 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.515885 32098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.515980 32098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:46.516080 32323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.516134 32323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:46.520466 32205 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 14:30:46.520603 32205 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 14:30:46.520638 32205 tablet_replica.cc:330] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: stopping tablet replica
I20260811 14:30:46.523721 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:36245
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:42483
--webserver_port=45049
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:30:46.534629 32205 master.cc:552] Master@127.23.186.252:43991 shutdown complete.
I20260811 14:30:46.534718 32205 master_runner.cc:305] Clearing existing system tablet
I20260811 14:30:46.534868 32205 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.534930 32205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.535600 32205 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:46.536149 32338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.536309 32205 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.536438 32205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:46.536482 32205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
uuid: "e431a103e3264c75ae234353c0158cc6"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "4b8d7e5b7ff7a377f85be86ce276f708"
server_key_iv: "32d73b758acfd346ebad9d8742b9a65f"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.537245 32205 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:46.538700 32205 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 14:30:46.538786 32205 log.cc:1196] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 14:30:46.539021 32205 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Deleting consensus metadata
I20260811 14:30:46.539168 32205 master_runner.cc:315] Copying system tablet from 127.23.186.254:42483
I20260811 14:30:46.539880 32205 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: tablet copy: Beginning tablet copy session from remote peer at address 127.23.186.254:42483
I20260811 14:30:46.546685 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.540299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54914 (local address 127.23.186.254:42483)
0811 14:30:46.540354 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.540356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.540487 (+ 131us) server_negotiation.cc:407] Connection header received
0811 14:30:46.540561 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.540563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.540575 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.540626 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.540990 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.541508 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.542400 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.542601 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.542681 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.543836 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.543837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.543837 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.543845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.545420 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.545815 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.545816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.545816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.545881 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.546101 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.546102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.546102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.546232 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.546312 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.546483 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.546577 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:30:46.546706 32345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.540249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:54914)
0811 14:30:46.540401 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.540423 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.540503 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.540629 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.540630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.540849 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.540905 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.540906 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.541595 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.541597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.542260 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.542262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.542679 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.542680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.542748 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.543290 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.543295 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.543723 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.545540 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.545541 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.545541 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.545727 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.545965 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.545966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.545967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.546036 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.546304 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.546305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.546354 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.546454 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.546574 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":72,"threads_started":1}
I20260811 14:30:46.547408 31446 tablet_copy_service.cc:140] P 760dc8b205a441b0bf16670e8dd716a7: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e431a103e3264c75ae234353c0158cc6 ({username='oryx', principal='oryx/127.23.186.252@KRBTEST.COM'} at 127.0.0.1:54914)
I20260811 14:30:46.547493 31446 tablet_copy_service.cc:161] P 760dc8b205a441b0bf16670e8dd716a7: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e431a103e3264c75ae234353c0158cc6 at {username='oryx', principal='oryx/127.23.186.252@KRBTEST.COM'} at 127.0.0.1:54914: session id = e431a103e3264c75ae234353c0158cc6-00000000000000000000000000000000
I20260811 14:30:46.548028 31446 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 14:30:46.548617 32205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 14:30:46.549674 32205 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: tablet copy: Starting download of 0 data blocks...
I20260811 14:30:46.549860 32205 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: tablet copy: Starting download of 1 WAL segments...
I20260811 14:30:46.550604 32205 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 14:30:46.551438 31446 tablet_copy_service.cc:342] P 760dc8b205a441b0bf16670e8dd716a7: Request end of tablet copy session e431a103e3264c75ae234353c0158cc6-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.23.186.252@KRBTEST.COM'} at 127.0.0.1:54914
I20260811 14:30:46.551517 31446 tablet_copy_service.cc:433] P 760dc8b205a441b0bf16670e8dd716a7: ending tablet copy session e431a103e3264c75ae234353c0158cc6-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e431a103e3264c75ae234353c0158cc6
I20260811 14:30:46.554116 32205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:46.555326 32346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.555563 32349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.555330 32347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.555768 32205 server_base.cc:1034] running on GCE node
I20260811 14:30:46.555851 32205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:46.556027 32205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:46.557159 32205 hybrid_clock.cc:648] HybridClock initialized: now 1786458646557158 us; error 21 us; skew 500 ppm
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:46.558657 32205 init.cc:377] Logged in from keytab as oryx/127.23.186.252@KRBTEST.COM (short username oryx)
I20260811 14:30:46.558977 32205 webserver.cc:460] Webserver started at http://127.23.186.252:40739/ using document root <none> and password file <none>
I20260811 14:30:46.559118 32205 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.559156 32205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.559824 32205 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:46.560395 32356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.560550 32205 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.560603 32205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
uuid: "e431a103e3264c75ae234353c0158cc6"
format_stamp: "Formatted at 2026-08-11 14:30:46 on dist-test-slave-cbsf"
server_key: "4b8d7e5b7ff7a377f85be86ce276f708"
server_key_iv: "32d73b758acfd346ebad9d8742b9a65f"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.560670 32205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.575783 32205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:46.577662 32205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:46.581496 32205 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.252:43991
I20260811 14:30:46.581511 32416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.252:43991 every 8 connection(s)
I20260811 14:30:46.581872 32205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
I20260811 14:30:46.582716 32418 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:30:46.583181 32418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Bootstrap starting.
I20260811 14:30:46.585222 32418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:30:46.585462 32418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Bootstrap complete.
I20260811 14:30:46.585697 32418 raft_consensus.cc:348] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } }
I20260811 14:30:46.585793 32418 raft_consensus.cc:729] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e431a103e3264c75ae234353c0158cc6, State: Initialized, Role: LEARNER
I20260811 14:30:46.585841 32418 consensus_queue.cc:260] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } }
I20260811 14:30:46.586057 32418 sys_catalog.cc:564] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:46.586045 32422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.586266 32422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: LEARNER
I20260811 14:30:46.587970 32433 catalog_manager.cc:1259] Loaded cluster ID: a1e47cbad435406387ae424fea8d7e30
I20260811 14:30:46.588058 32433 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: loading cluster ID for follower catalog manager: success
I20260811 14:30:46.589277 32433 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: acquiring CA information for follower catalog manager: success
I20260811 14:30:46.589622 32433 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 14:30:46.647825 32336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:46.648110 32336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:46.648155 32336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:46.649523 32336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:46.649601 32336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:46.649650 32336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:46.651458 32336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:36245
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=45049
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:46.651762 32336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:46.652022 32336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:46.657307 32440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.657357 32443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.657305 32439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.657917 32336 server_base.cc:1034] running on GCE node
I20260811 14:30:46.658146 32336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:46.658453 32336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:46.659616 32336 hybrid_clock.cc:648] HybridClock initialized: now 1786458646659606 us; error 42 us; skew 500 ppm
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:46.662595 32336 init.cc:377] Logged in from keytab as oryx/127.23.186.193@KRBTEST.COM (short username oryx)
I20260811 14:30:46.663172 32336 webserver.cc:460] Webserver started at http://127.23.186.193:45049/ using document root <none> and password file <none>
I20260811 14:30:46.663434 32336 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.663515 32336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.665302 32336 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:30:46.666046 32451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.666222 32336 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:46.666317 32336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "65c9aa5e5d004db183da0827fc6d75c5"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "8f8c5c700c3b7d54f6e43842485a1720"
server_key_iv: "eed6f01a01f3afa32384933149856ad7"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.666682 32336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.683241 32336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:46.685875 32336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:46.686045 32336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:46.686386 32336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:46.686759 32336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:46.686822 32336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.686856 32336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:46.686878 32336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.694551 32336 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:36245
I20260811 14:30:46.694600 32587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:36245 every 8 connection(s)
I20260811 14:30:46.695041 32336 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:46.701481 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 32336
I20260811 14:30:46.701669 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31662
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.252@KRBTEST.COM
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
I20260811 14:30:46.708480 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.696294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:50729 (local address 127.23.186.253:34957)
0811 14:30:46.696392 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.696395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.696403 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:46.697093 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.697095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.697109 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.697183 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.698103 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.698879 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700179 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.700478 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700555 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.704851 (+ 4296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.704853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.704854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.704865 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.706629 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707250 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.707252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.707253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.707321 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707648 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.707649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.707649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.707816 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.707861 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.708279 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.708378 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 14:30:46.708480 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.696221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:54321 (local address 127.23.186.254:42483)
0811 14:30:46.696345 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.696347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.696359 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:46.697254 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.697256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.697273 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.697326 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.698102 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.698880 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.699981 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.700261 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700330 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.704854 (+ 4524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.704856 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.704856 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.704866 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.706777 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707287 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.707289 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.707289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.707349 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707648 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.707649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.707649 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.707799 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.707848 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.708284 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.708378 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 14:30:46.708746 32594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.695919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.23.186.193:48395)
0811 14:30:46.696316 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.696325 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.696907 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.697566 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.697567 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.697753 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.697982 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.697986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.698917 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.698919 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.700078 (+ 1159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.700081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.700603 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.700679 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.701239 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.701252 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.704119 (+ 2867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.706590 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.706602 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.706605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.707147 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.707440 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707442 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.707442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.707565 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.707894 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.707902 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.708108 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.708223 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.708437 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":311,"thread_start_us":66,"threads_started":1}
I20260811 14:30:46.709096 32596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.695919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:54321)
0811 14:30:46.696271 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.696284 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.696884 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.697360 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.697361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.697761 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.697982 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.697989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.698984 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.698985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.699863 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.699865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.700349 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.700499 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.701271 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.701275 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.704722 (+ 3447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.706864 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.706864 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.706865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.707199 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.707425 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707425 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.707426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.707563 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.707901 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.707903 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.708108 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.708582 (+ 474us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.708759 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":36,"thread_start_us":97,"threads_started":1}
I20260811 14:30:46.709174 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.696033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:48395 (local address 127.23.186.252:43991)
0811 14:30:46.696427 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.696429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.696439 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:46.697004 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.697006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.697023 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.697097 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.698094 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.698794 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700190 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.700478 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700543 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.704279 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.704282 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.704282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.704295 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.706479 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707239 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.707241 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.707241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.707350 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707641 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.707642 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.707642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.707802 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.707866 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.708897 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.709024 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":62,"threads_started":1}
I20260811 14:30:46.709683 32588 heartbeater.cc:347] Connected to a master server at 127.23.186.252:43991
I20260811 14:30:46.709794 32588 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:46.709995 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.695919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.23.186.193:50729)
0811 14:30:46.696136 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.696186 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.696893 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.697223 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.697228 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.697755 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.697982 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.697989 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.698962 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.698963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.700078 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.700081 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.700570 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.700571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.700647 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.701398 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.701404 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.704722 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.706740 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.706742 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.706743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.707148 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.707402 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.707403 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.707403 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.707563 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.707910 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.707911 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.708108 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.709576 (+ 1468us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.709707 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":187,"thread_start_us":99,"threads_started":1}
I20260811 14:30:46.710029 32588 heartbeater.cc:511] Master 127.23.186.252:43991 requested a full tablet report, sending...
I20260811 14:30:46.710626 32372 ts_manager.cc:194] Registered new tserver with Master: 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:46.713331 32590 heartbeater.cc:347] Connected to a master server at 127.23.186.253:34957
I20260811 14:30:46.713369 32589 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42483
I20260811 14:30:46.713403 32590 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:46.713428 32589 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:46.713498 32590 heartbeater.cc:511] Master 127.23.186.253:34957 requested a full tablet report, sending...
I20260811 14:30:46.713508 32589 heartbeater.cc:511] Master 127.23.186.254:42483 requested a full tablet report, sending...
I20260811 14:30:46.713819 31417 ts_manager.cc:194] Re-registered known tserver with Master: 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:46.713945 32143 ts_manager.cc:194] Registered new tserver with Master: 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:46.714409 31417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:54321
I20260811 14:30:46.714464 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:33035
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:42483
--webserver_port=34645
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:46.804136 32597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:46.804382 32597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:46.804409 32597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:46.805680 32597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:46.805729 32597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:46.805758 32597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:46.807276 32597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:33035
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=34645
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32597
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:46.807493 32597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:46.807691 32597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:46.813104 32602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.813104 32605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:46.813169 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:46.813231 32597 server_base.cc:1034] running on GCE node
I20260811 14:30:46.813577 32597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:46.813797 32597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:46.814997 32597 hybrid_clock.cc:648] HybridClock initialized: now 1786458646814987 us; error 28 us; skew 500 ppm
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:46.817804 32597 init.cc:377] Logged in from keytab as oryx/127.23.186.194@KRBTEST.COM (short username oryx)
I20260811 14:30:46.818216 32597 webserver.cc:460] Webserver started at http://127.23.186.194:34645/ using document root <none> and password file <none>
I20260811 14:30:46.818424 32597 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:46.818478 32597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:46.819777 32597 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:46.820397 32614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.820581 32597 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:30:46.820734 32597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "07b4360466d842848a1fb5033a4ccf22"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "30a40e78ecbb0c691fc949eb569576e0"
server_key_iv: "fe879d9da0e67af5b74471ae1006e457"
server_key_version: "encryptionkey@0"
I20260811 14:30:46.821105 32597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:46.868199 32597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:46.870653 32597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:46.870850 32597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:46.871122 32597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:46.871476 32597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:46.871541 32597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.871583 32597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:46.871613 32597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:46.877667 32597 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33035
I20260811 14:30:46.877714 32747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33035 every 8 connection(s)
I20260811 14:30:46.878193 32597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:46.880920 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 32597
I20260811 14:30:46.881091 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31821
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.252@KRBTEST.COM
I20260811 14:30:46.892453 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.879319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:50875 (local address 127.23.186.252:43991)
0811 14:30:46.879366 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.879368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.879380 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:46.879987 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.879989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.880002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.880053 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.883667 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.884368 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.885905 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.886199 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.886257 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.889365 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.889367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.889367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.889377 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.890885 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.891362 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.891364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.891365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.891441 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.891720 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.891721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.891721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.891852 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.891934 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.892242 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.892344 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:46.892642 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.878984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.23.186.194:50875)
0811 14:30:46.879214 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.879261 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.879858 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.880138 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.880143 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.880609 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.880973 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.880980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.884498 (+ 3518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.884499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.885788 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.885791 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.886396 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.886396 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.886549 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.887354 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.887365 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.889238 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.891009 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.891019 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.891022 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.891269 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.891516 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.891517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.891518 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.891644 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.891932 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.891940 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.892097 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.892208 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.892389 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":2,"thread_start_us":100,"threads_started":1}
I20260811 14:30:46.900913 32749 heartbeater.cc:347] Connected to a master server at 127.23.186.252:43991
I20260811 14:30:46.901079 32749 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:46.901325 32749 heartbeater.cc:511] Master 127.23.186.252:43991 requested a full tablet report, sending...
I20260811 14:30:46.902511 32372 ts_manager.cc:194] Registered new tserver with Master: 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035)
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
Aug 11 14:30:46 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458646, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:46.907498 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:43197
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:42483
--webserver_port=34985
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:30:46.909278 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.878985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.23.186.194:44277)
0811 14:30:46.879381 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.879391 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.879881 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.880625 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.880626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.880858 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.880973 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.881050 (+ 77us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.900852 (+ 19802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.900853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.902066 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.902071 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.902737 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.902738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.902834 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.903454 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.903461 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.905273 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.907340 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.907341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.907342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.907885 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.908122 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.908123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.908123 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.908187 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.908881 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.908882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.908936 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.909008 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.909130 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":200,"thread_start_us":73,"threads_started":1}
I20260811 14:30:46.909690 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.878998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:44277 (local address 127.23.186.253:34957)
0811 14:30:46.879077 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.879079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.879456 (+ 377us) server_negotiation.cc:407] Connection header received
0811 14:30:46.880069 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.880070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.880082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.880144 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.895835 (+ 15691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.896933 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.902203 (+ 5270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.902564 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.902634 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.905393 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.905395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.905396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.905406 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.907235 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.907984 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.907986 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.907986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.908052 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.908286 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.908288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.908288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.908764 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.908846 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.909084 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.909194 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:30:46.910228 32751 heartbeater.cc:347] Connected to a master server at 127.23.186.253:34957
I20260811 14:30:46.910300 32751 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:46.910383 32751 heartbeater.cc:511] Master 127.23.186.253:34957 requested a full tablet report, sending...
I20260811 14:30:46.910665 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.878985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:48201)
0811 14:30:46.879330 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.879399 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.879881 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.880322 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.880323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.880610 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:46.881119 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.881121 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.903949 (+ 22828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.903950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.904669 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.904672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.905245 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.905246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.905324 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.905823 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:46.905827 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:46.907520 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:46.909525 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.909526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.909526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.909696 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.909941 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:46.909942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:46.909942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:46.910003 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:46.910326 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:46.910327 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:46.910372 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.910448 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.910566 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":392,"thread_start_us":101,"threads_started":1}
I20260811 14:30:46.910725 32143 ts_manager.cc:194] Registered new tserver with Master: 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035)
I20260811 14:30:46.911044 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.879103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:48201 (local address 127.23.186.254:42483)
0811 14:30:46.879153 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.879155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.879480 (+ 325us) server_negotiation.cc:407] Connection header received
0811 14:30:46.880209 (+ 729us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.880210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.880226 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.880281 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:46.903030 (+ 22749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.903844 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.904876 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.905143 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.905239 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.907619 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:46.907621 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:46.907621 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:46.907630 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:46.909435 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.909793 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.909794 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.909794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.909864 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:46.910077 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:46.910078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:46.910078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:46.910230 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:46.910296 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.910820 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.910908 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:30:46.911547 32750 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42483
I20260811 14:30:46.911597 32750 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:46.911679 32750 heartbeater.cc:511] Master 127.23.186.254:42483 requested a full tablet report, sending...
I20260811 14:30:46.911973 31417 ts_manager.cc:194] Re-registered known tserver with Master: 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035)
I20260811 14:30:46.912315 31417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:48201
I20260811 14:30:46.954941 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.952144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.0.0.1:55366)
0811 14:30:46.952211 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:46.952231 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:46.952308 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:46.952487 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:46.952489 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:46.952491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:46.952556 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.952558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.953316 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.953318 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.954072 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:46.954074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.954670 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.954671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:46.954734 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.954738 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:46.954804 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 14:30:46.954858 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:30:46.954941 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:46.952230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55366 (local address 127.23.186.252:43991)
0811 14:30:46.952309 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:46.952311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:46.952322 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:46.952376 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:46.952378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:46.952408 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:46.952482 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:46.952635 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.953215 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.954171 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:46.954578 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:46.954670 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:46.954750 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:46.954812 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:30:46.954855 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:30:46.955336 32389 raft_consensus.cc:1203] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 14:30:46.955506 32422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 760dc8b205a441b0bf16670e8dd716a7. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:46.955638 32422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: LEARNER
W20260811 14:30:47.002926 32758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:47.003147 32758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:47.003172 32758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:47.004447 32758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:47.004489 32758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:47.004539 32758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:47.006126 32758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:43197
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=34985
--tserver_master_addrs=127.23.186.254:42483,127.23.186.253:34957,127.23.186.252:43991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:47.006385 32758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:47.006635 32758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:47.011831 32766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:47.011843 301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:47.011982 32767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:47.012185 32758 server_base.cc:1034] running on GCE node
I20260811 14:30:47.012401 32758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:47.012639 32758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:47.013800 32758 hybrid_clock.cc:648] HybridClock initialized: now 1786458647013771 us; error 38 us; skew 500 ppm
Aug 11 14:30:47 dist-test-slave-cbsf krb5kdc[31364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458647, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:47.017001 32758 init.cc:377] Logged in from keytab as oryx/127.23.186.195@KRBTEST.COM (short username oryx)
I20260811 14:30:47.017486 32758 webserver.cc:460] Webserver started at http://127.23.186.195:34985/ using document root <none> and password file <none>
I20260811 14:30:47.017729 32758 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:47.017792 32758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:47.019162 32758 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:47.019731 309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:47.019941 32758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:47.020037 32758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "c599349c89f047c2975dbea16464dca1"
format_stamp: "Formatted at 2026-08-11 14:30:45 on dist-test-slave-cbsf"
server_key: "78853d13fbbeed03471548dab7c70f63"
server_key_iv: "0f6972d017e257b1664231356c9e6472"
server_key_version: "encryptionkey@0"
I20260811 14:30:47.020550 32758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:47.036540 32758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:47.039423 32758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:47.039626 32758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:47.039912 32758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:47.040268 32758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:47.040328 32758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:47.040375 32758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:47.040406 32758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:47.047441 32758 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:43197
I20260811 14:30:47.047545 445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:43197 every 8 connection(s)
I20260811 14:30:47.047945 32758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:47.055088 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 32758
Aug 11 14:30:47 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458647, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
Aug 11 14:30:47 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458647, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.252@KRBTEST.COM
Aug 11 14:30:47 dist-test-slave-cbsf krb5kdc[31364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458647, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.253@KRBTEST.COM
I20260811 14:30:47.061216 453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.048860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.23.186.195:37961)
0811 14:30:47.049289 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.049296 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.050336 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.050629 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.050630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.051168 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:47.051398 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.051404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.052408 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.052409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.053371 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.053373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.054446 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.054448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.054532 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.054941 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:47.054945 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:47.057143 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:47.059353 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:47.059363 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:47.059367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:47.059698 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:47.059930 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:47.059931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:47.059931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:47.060076 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:47.060396 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:47.060403 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:47.060597 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.060745 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.060948 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":83,"thread_start_us":77,"threads_started":1}
I20260811 14:30:47.061233 31467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.049131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:55443 (local address 127.23.186.254:42483)
0811 14:30:47.049183 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.049185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.049196 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:47.050473 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.050475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.050489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.050524 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.051518 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.052301 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053466 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.053758 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053824 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.058275 (+ 4451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.058276 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.058276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.058285 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.059936 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.060317 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.060318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.060319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.060376 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.060564 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.060565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.060565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.060755 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.060811 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.061053 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.061148 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:30:47.061547 32595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.048803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:37961 (local address 127.23.186.252:43991)
0811 14:30:47.048888 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.048890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.049362 (+ 472us) server_negotiation.cc:407] Connection header received
0811 14:30:47.050473 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.050475 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.050489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.050524 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.051519 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.052306 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053653 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.053951 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.054449 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.057274 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.057276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.057276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.057285 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.059256 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.059794 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.059795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.059796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.059860 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.060152 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.060153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.060154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.060306 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.060368 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.061375 (+ 1007us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.061458 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 14:30:47.061565 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.048803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:55443)
0811 14:30:47.048997 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.049043 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.050333 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.050625 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.050629 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.051181 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:47.051398 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.051404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.052408 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.052409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.053370 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.053373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.053844 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.053914 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.054403 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:47.054407 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:47.058170 (+ 3763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:47.060014 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:47.060015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:47.060015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:47.060232 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:47.060442 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:47.060443 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:47.060443 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:47.060496 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:47.060869 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:47.060870 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:47.060910 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.061349 (+ 439us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.061458 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":9,"thread_start_us":116,"threads_started":1}
I20260811 14:30:47.061983 448 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42483
I20260811 14:30:47.062083 448 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:47.062244 448 heartbeater.cc:511] Master 127.23.186.254:42483 requested a full tablet report, sending...
I20260811 14:30:47.062707 31417 ts_manager.cc:194] Re-registered known tserver with Master: c599349c89f047c2975dbea16464dca1 (127.23.186.195:43197)
I20260811 14:30:47.062757 446 heartbeater.cc:347] Connected to a master server at 127.23.186.252:43991
I20260811 14:30:47.062796 446 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:47.062856 446 heartbeater.cc:511] Master 127.23.186.252:43991 requested a full tablet report, sending...
I20260811 14:30:47.063238 31417 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:55443
I20260811 14:30:47.063503 32372 ts_manager.cc:194] Registered new tserver with Master: c599349c89f047c2975dbea16464dca1 (127.23.186.195:43197)
I20260811 14:30:47.063959 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.048806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.23.186.195:36501)
0811 14:30:47.049168 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.049180 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.050335 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.050624 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.050628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.051136 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:47.051396 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.051400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.052155 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.052156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.053143 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.053146 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.053589 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.053710 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.054315 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:47.054327 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:47.057831 (+ 3504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:47.060864 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:47.060865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:47.060866 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:47.061141 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:47.063104 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:47.063105 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:47.063105 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:47.063177 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:47.063527 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:47.063528 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:47.063579 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.063656 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.063813 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":23,"thread_start_us":138,"threads_started":1}
I20260811 14:30:47.064808 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.049148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:36501 (local address 127.23.186.253:34957)
0811 14:30:47.049192 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.049195 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.049242 (+ 47us) server_negotiation.cc:407] Connection header received
0811 14:30:47.050456 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.050457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.050471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.050533 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.051513 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.052025 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053243 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.053498 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.053570 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.057982 (+ 4412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.057984 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.057984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.057991 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.060767 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.061240 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.061242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.061242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.061307 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.063272 (+ 1965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.063274 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.063274 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.063448 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.063521 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.064552 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.064685 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:30:47.065107 449 heartbeater.cc:347] Connected to a master server at 127.23.186.253:34957
I20260811 14:30:47.065165 449 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:47.065253 449 heartbeater.cc:511] Master 127.23.186.253:34957 requested a full tablet report, sending...
I20260811 14:30:47.065346 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.056524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55370 (local address 127.23.186.252:43991)
0811 14:30:47.056810 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.056812 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.056967 (+ 155us) server_negotiation.cc:407] Connection header received
0811 14:30:47.056991 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.056992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.057004 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.057059 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.057777 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.058587 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.059746 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.060058 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.060126 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.061757 (+ 1631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.061759 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.061759 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.061767 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.063634 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.064033 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.064035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.064035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.064140 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.064376 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.064377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.064377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.064537 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.064736 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.064848 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.065218 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":71,"threads_started":1}
I20260811 14:30:47.065577 32143 ts_manager.cc:194] Registered new tserver with Master: c599349c89f047c2975dbea16464dca1 (127.23.186.195:43197)
I20260811 14:30:47.071116 459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.056277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54928 (local address 127.23.186.254:42483)
0811 14:30:47.056543 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.056545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.056761 (+ 216us) server_negotiation.cc:407] Connection header received
0811 14:30:47.056884 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.056885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.056899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.056967 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.064270 (+ 7303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.065393 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.066323 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.066647 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.066739 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.067972 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.067973 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.067974 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.067983 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.069897 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.070302 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.070303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.070303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.070362 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.070580 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.070580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.070581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.070731 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.070837 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.070965 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.071045 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":65,"threads_started":1}
I20260811 14:30:47.072124 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.057858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35940 (local address 127.23.186.253:34957)
0811 14:30:47.064898 (+ 7040us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.064900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.064908 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:47.064930 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.064931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.064941 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.065000 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.066029 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.066705 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.067441 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.067660 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.067739 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.069107 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.069109 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.069109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.069117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.070743 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.071207 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.071208 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.071209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.071267 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.071479 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.071481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.071481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.071662 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.071769 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.071910 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.072005 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7021,"thread_start_us":77,"threads_started":1}
I20260811 14:30:47.073155 31417 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54928:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:47.074747 31417 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:47.075533 32323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: NON_VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: true } } }
I20260811 14:30:47.075642 32323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:47.076809 32323 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: attempting to promote NON_VOTER e431a103e3264c75ae234353c0158cc6 to VOTER
I20260811 14:30:47.076974 32323 consensus_queue.cc:237] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:47.077417 32154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Refusing update from remote peer 760dc8b205a441b0bf16670e8dd716a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 14:30:47.077512 32389 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 LEARNER]: Refusing update from remote peer 760dc8b205a441b0bf16670e8dd716a7: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 14:30:47.077690 469 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 14:30:47.077869 468 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 14:30:47.078691 469 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e431a103e3264c75ae234353c0158cc6 (127.23.186.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.078899 32389 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e431a103e3264c75ae234353c0158cc6 (127.23.186.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.079826 469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.079852 466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.079921 469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:47.079941 466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:47.079975 469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.080031 469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:47.080045 32422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.080147 32422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:30:47.080335 32154 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, e431a103e3264c75ae234353c0158cc6 (127.23.186.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.081516 479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "760dc8b205a441b0bf16670e8dd716a7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:47.081663 479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:30:47.086812 32097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.081920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:36245 (local address 127.0.0.1:39136)
0811 14:30:47.081978 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.081992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.082036 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.083061 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.083062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.083065 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:47.083122 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.083125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.084584 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.084585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.085477 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.085480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.086538 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.086539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.086603 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.086606 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.086661 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.086711 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":32}
I20260811 14:30:47.086930 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.082446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:43197 (local address 127.0.0.1:54224)
0811 14:30:47.083140 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.083150 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.083190 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.083381 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.083382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.083384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:47.083443 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.083445 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.084306 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.084308 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.085621 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.085624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.086674 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.086675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.086723 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.086726 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.086780 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.086814 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":73,"threads_started":1}
I20260811 14:30:47.087033 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.082525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54224 (local address 127.23.186.195:43197)
0811 14:30:47.083214 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.083218 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.083229 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:47.083265 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.083267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.083280 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.083382 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:47.083500 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.084211 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.085743 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.086459 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.086574 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.086640 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.086848 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.086903 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":128,"threads_started":1}
I20260811 14:30:47.087288 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.082056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39136 (local address 127.23.186.193:36245)
0811 14:30:47.082861 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.082865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.082878 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:30:47.082929 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.082932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.082952 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.083061 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:47.083225 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.084478 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.085579 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.086445 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.086552 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.086622 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.087113 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.087182 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":86,"threads_started":1}
I20260811 14:30:47.087826 368 tablet_service.cc:1467] Processing CreateTablet for tablet 628a4d74b2df4c2f9ebcb134b4aaae63 (DEFAULT_TABLE table=test-table [id=8123fd37782048a28349b14a706a3f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:47.087895 32510 tablet_service.cc:1467] Processing CreateTablet for tablet 628a4d74b2df4c2f9ebcb134b4aaae63 (DEFAULT_TABLE table=test-table [id=8123fd37782048a28349b14a706a3f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:47.088285 368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628a4d74b2df4c2f9ebcb134b4aaae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:47.088310 32510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628a4d74b2df4c2f9ebcb134b4aaae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:47.089067 484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.081967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33035 (local address 127.0.0.1:37460)
0811 14:30:47.082991 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.083001 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.083058 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.083330 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.083331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.083334 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:47.083387 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.083389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.084650 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.084651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.085822 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.085824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.088794 (+ 2970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.088795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.088851 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.088855 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.088933 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.088982 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"mutex_wait_us":51,"thread_start_us":103,"threads_started":1}
I20260811 14:30:47.089080 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.082256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37460 (local address 127.23.186.194:33035)
0811 14:30:47.082428 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.082433 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.083075 (+ 642us) server_negotiation.cc:407] Connection header received
0811 14:30:47.083136 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.083139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.083158 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.083240 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:47.083470 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.084568 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.085900 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.086714 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.087361 (+ 647us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.087456 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.088954 (+ 1498us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.089002 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":85,"threads_started":1}
I20260811 14:30:47.089931 32672 tablet_service.cc:1467] Processing CreateTablet for tablet 628a4d74b2df4c2f9ebcb134b4aaae63 (DEFAULT_TABLE table=test-table [id=8123fd37782048a28349b14a706a3f56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:47.090372 32672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 628a4d74b2df4c2f9ebcb134b4aaae63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:47.092316 486 tablet_bootstrap.cc:492] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Bootstrap starting.
I20260811 14:30:47.092846 487 tablet_bootstrap.cc:492] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Bootstrap starting.
I20260811 14:30:47.093133 486 tablet_bootstrap.cc:654] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:47.093422 487 tablet_bootstrap.cc:654] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:47.093422 488 tablet_bootstrap.cc:492] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Bootstrap starting.
I20260811 14:30:47.093772 487 log.cc:824] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:47.093909 486 log.cc:824] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:47.094079 488 tablet_bootstrap.cc:654] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:47.094456 488 log.cc:824] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:47.094663 487 tablet_bootstrap.cc:492] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: No bootstrap required, opened a new log
I20260811 14:30:47.094784 487 ts_tablet_manager.cc:1397] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:47.094969 486 tablet_bootstrap.cc:492] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: No bootstrap required, opened a new log
I20260811 14:30:47.095088 486 ts_tablet_manager.cc:1397] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:30:47.095325 488 tablet_bootstrap.cc:492] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: No bootstrap required, opened a new log
I20260811 14:30:47.095396 488 ts_tablet_manager.cc:1397] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:47.096446 487 raft_consensus.cc:348] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.096618 487 raft_consensus.cc:374] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:47.096666 487 raft_consensus.cc:729] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c9aa5e5d004db183da0827fc6d75c5, State: Initialized, Role: FOLLOWER
I20260811 14:30:47.096833 487 consensus_queue.cc:260] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.097187 487 ts_tablet_manager.cc:1428] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:30:47.097425 32589 heartbeater.cc:503] Master 127.23.186.254:42483 was elected leader, sending a full tablet report...
I20260811 14:30:47.098277 488 raft_consensus.cc:348] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.098277 486 raft_consensus.cc:348] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.098448 486 raft_consensus.cc:374] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:47.098448 488 raft_consensus.cc:374] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:47.098484 486 raft_consensus.cc:729] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c599349c89f047c2975dbea16464dca1, State: Initialized, Role: FOLLOWER
I20260811 14:30:47.098484 488 raft_consensus.cc:729] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b4360466d842848a1fb5033a4ccf22, State: Initialized, Role: FOLLOWER
I20260811 14:30:47.098627 486 consensus_queue.cc:260] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.098636 488 consensus_queue.cc:260] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.098912 486 ts_tablet_manager.cc:1428] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 14:30:47.098912 488 ts_tablet_manager.cc:1428] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:30:47.099272 448 heartbeater.cc:503] Master 127.23.186.254:42483 was elected leader, sending a full tablet report...
I20260811 14:30:47.099684 32750 heartbeater.cc:503] Master 127.23.186.254:42483 was elected leader, sending a full tablet report...
I20260811 14:30:47.120949 496 raft_consensus.cc:482] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:47.121137 496 raft_consensus.cc:504] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.121528 496 leader_election.cc:288] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035), 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:47.125643 482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.121833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:51981 (local address 127.23.186.193:36245)
0811 14:30:47.121882 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.121884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.121895 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:47.121927 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.121928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.121938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.122008 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:47.122268 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.122998 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.124544 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.125185 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.125268 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.125278 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.125520 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.125569 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:47.125643 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.121838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:45929 (local address 127.23.186.194:33035)
0811 14:30:47.121888 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.121890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.121897 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:47.121927 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.121928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.121938 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.122008 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:47.122214 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.123008 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.124570 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.125185 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.125268 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.125278 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.125524 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.125579 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:30:47.125674 451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.121721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:36245 (local address 127.23.186.195:51981)
0811 14:30:47.121791 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.121805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.121851 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.122027 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.122029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.122030 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:47.122194 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.122196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.123102 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.123103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.124442 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.124446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.125326 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.125327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.125421 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.125428 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.125501 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.125561 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":125}
I20260811 14:30:47.125674 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.121721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33035 (local address 127.23.186.195:45929)
0811 14:30:47.121801 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:47.121810 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:47.121851 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:47.122027 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:47.122029 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:47.122030 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:47.122139 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.122141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.123102 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.123103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.124442 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:47.124446 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.125326 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.125327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:47.125421 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.125428 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:30:47.125500 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:30:47.125561 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":45}
I20260811 14:30:47.126174 32695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628a4d74b2df4c2f9ebcb134b4aaae63" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b4360466d842848a1fb5033a4ccf22" is_pre_election: true
I20260811 14:30:47.126291 32534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628a4d74b2df4c2f9ebcb134b4aaae63" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9aa5e5d004db183da0827fc6d75c5" is_pre_election: true
I20260811 14:30:47.126376 32695 raft_consensus.cc:2454] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 0.
I20260811 14:30:47.126427 32534 raft_consensus.cc:2454] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 0.
I20260811 14:30:47.126648 311 leader_election.cc:302] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b4360466d842848a1fb5033a4ccf22, c599349c89f047c2975dbea16464dca1; no voters:
I20260811 14:30:47.126821 496 raft_consensus.cc:2781] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:47.126904 496 raft_consensus.cc:482] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:47.126924 496 raft_consensus.cc:3037] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:47.127866 496 raft_consensus.cc:504] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.128029 496 leader_election.cc:288] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 election: Requested vote from peers 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035), 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:47.128222 32534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628a4d74b2df4c2f9ebcb134b4aaae63" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9aa5e5d004db183da0827fc6d75c5"
I20260811 14:30:47.128242 32695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "628a4d74b2df4c2f9ebcb134b4aaae63" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b4360466d842848a1fb5033a4ccf22"
I20260811 14:30:47.128312 32534 raft_consensus.cc:3037] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:47.128312 32695 raft_consensus.cc:3037] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Advancing to term 1
W20260811 14:30:47.128950 32753 tablet.cc:2369] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:47.129262 32695 raft_consensus.cc:2454] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 1.
I20260811 14:30:47.129262 32534 raft_consensus.cc:2454] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 1.
I20260811 14:30:47.129462 313 leader_election.cc:302] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65c9aa5e5d004db183da0827fc6d75c5, c599349c89f047c2975dbea16464dca1; no voters:
I20260811 14:30:47.129559 496 raft_consensus.cc:2781] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:47.129757 496 raft_consensus.cc:686] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 1 LEADER]: Becoming Leader. State: Replica: c599349c89f047c2975dbea16464dca1, State: Running, Role: LEADER
I20260811 14:30:47.129871 496 consensus_queue.cc:237] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:47.130740 31415 catalog_manager.cc:5495] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c599349c89f047c2975dbea16464dca1 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "c599349c89f047c2975dbea16464dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } health_report { overall_health: HEALTHY } } }
I20260811 14:30:47.144446 485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.142206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54234 (local address 127.23.186.195:43197)
0811 14:30:47.142257 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.142259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.142268 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:47.142306 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.142306 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.142324 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.142389 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:47.142496 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.143027 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.143745 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.143933 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.143986 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.144148 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:47.144255 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:47.144317 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.144347 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.144367 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:47.146554 32534 raft_consensus.cc:1261] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Refusing update from remote peer c599349c89f047c2975dbea16464dca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:47.146559 32695 raft_consensus.cc:1261] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Refusing update from remote peer c599349c89f047c2975dbea16464dca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:47.146941 496 consensus_queue.cc:1035] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:47.147114 498 consensus_queue.cc:1035] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:47.151087 504 mvcc.cc:205] Tried to move back new op lower bound from 7317334618708893696 to 7317334618644668416. Current Snapshot: MvccSnapshot[applied={T|T < 7317334618708893696}]
I20260811 14:30:47.152245 503 mvcc.cc:205] Tried to move back new op lower bound from 7317334618708893696 to 7317334618644668416. Current Snapshot: MvccSnapshot[applied={T|T < 7317334618708893696 or (T in {7317334618708893696})}]
I20260811 14:30:47.154594 31415 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:47.154707 31415 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54928:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:47.156587 31415 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 760dc8b205a441b0bf16670e8dd716a7: Sending DeleteTablet for 3 replicas of tablet 628a4d74b2df4c2f9ebcb134b4aaae63
I20260811 14:30:47.156988 32672 tablet_service.cc:1514] Processing DeleteTablet for tablet 628a4d74b2df4c2f9ebcb134b4aaae63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:47 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:37460
I20260811 14:30:47.156976 32510 tablet_service.cc:1514] Processing DeleteTablet for tablet 628a4d74b2df4c2f9ebcb134b4aaae63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:47 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:39136
I20260811 14:30:47.157016 368 tablet_service.cc:1514] Processing DeleteTablet for tablet 628a4d74b2df4c2f9ebcb134b4aaae63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:47 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:54224
I20260811 14:30:47.157271 512 tablet_replica.cc:330] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: stopping tablet replica
I20260811 14:30:47.157390 512 raft_consensus.cc:2229] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:47.157503 512 raft_consensus.cc:2258] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:47.157668 514 tablet_replica.cc:330] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: stopping tablet replica
I20260811 14:30:47.157759 514 raft_consensus.cc:2229] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:47.157904 514 raft_consensus.cc:2258] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:47.158178 512 ts_tablet_manager.cc:1905] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:47.158361 514 ts_tablet_manager.cc:1905] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:47.158385 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31379
I20260811 14:30:47.163767 512 ts_tablet_manager.cc:1918] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:47.163874 512 log.cc:1196] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/628a4d74b2df4c2f9ebcb134b4aaae63
I20260811 14:30:47.164007 514 ts_tablet_manager.cc:1918] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:47.164078 514 log.cc:1196] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/628a4d74b2df4c2f9ebcb134b4aaae63
I20260811 14:30:47.164194 512 ts_tablet_manager.cc:1939] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 65c9aa5e5d004db183da0827fc6d75c5: Deleting consensus metadata
I20260811 14:30:47.164368 514 ts_tablet_manager.cc:1939] T 628a4d74b2df4c2f9ebcb134b4aaae63 P c599349c89f047c2975dbea16464dca1: Deleting consensus metadata
W20260811 14:30:47.169365 311 connection.cc:535] server connection from 127.0.0.1:54224 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 14:30:47.169605 32453 connection.cc:535] server connection from 127.0.0.1:39136 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 14:30:47.171988 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.157844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55378 (local address 127.23.186.252:43991)
0811 14:30:47.157891 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.157893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.158313 (+ 420us) server_negotiation.cc:407] Connection header received
0811 14:30:47.158418 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.158419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.158434 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.158464 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.163862 (+ 5398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.164769 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.165935 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.166288 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.166333 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.168198 (+ 1865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.168200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.168200 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.168210 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.170289 (+ 2079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.170678 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.170679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.170680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.170737 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.170919 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.170921 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.170921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.171073 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.171199 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.171775 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.171876 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:47.176869 513 tablet_replica.cc:330] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: stopping tablet replica
I20260811 14:30:47.177053 513 raft_consensus.cc:2229] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:47.177240 513 raft_consensus.cc:2258] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:47.177845 513 ts_tablet_manager.cc:1905] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:47.179862 513 ts_tablet_manager.cc:1918] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:47.179945 513 log.cc:1196] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/628a4d74b2df4c2f9ebcb134b4aaae63
I20260811 14:30:47.180277 513 ts_tablet_manager.cc:1939] T 628a4d74b2df4c2f9ebcb134b4aaae63 P 07b4360466d842848a1fb5033a4ccf22: Deleting consensus metadata
I20260811 14:30:47.180481 32188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.171370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.23.186.253:34957)
0811 14:30:47.171419 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.171421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.171571 (+ 150us) server_negotiation.cc:407] Connection header received
0811 14:30:47.171692 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.171694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.171708 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.171767 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.173418 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.174134 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.174903 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.175136 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.175220 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.176634 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.176635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.176635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.176644 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.178949 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.179460 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.179462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.179462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.179527 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.179789 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.179790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.179790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.179935 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.180046 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.180301 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.180393 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:30:47.180508 461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:47.172390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55390 (local address 127.23.186.252:43991)
0811 14:30:47.172426 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:47.172428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:47.172435 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:47.172457 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:47.172458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:47.172468 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:47.172538 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:47.173271 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.174038 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.175246 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:47.175563 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:47.175603 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:47.177350 (+ 1747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:47.177351 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:47.177351 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:47.177359 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:47.179019 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.179500 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.179501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.179502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.179564 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:47.179792 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:47.179793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:47.179793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:47.179931 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:47.180005 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:47.180323 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 14:30:47.180412 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 14:30:47.180737 32615 connection.cc:441] server connection from 127.0.0.1:37460 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:37460 (request call id 1) could send its response
I20260811 14:30:48.152525 523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.152254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:47435)
0811 14:30:48.152380 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.152467 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":61,"threads_started":1}
I20260811 14:30:48.152570 522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.152221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:41967)
0811 14:30:48.152427 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.152519 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
W20260811 14:30:48.152776 448 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:42483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.152994 524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.152761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:56783)
0811 14:30:48.152874 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.152947 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":50,"threads_started":1}
W20260811 14:30:48.153110 32750 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:42483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
W20260811 14:30:48.153304 32589 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:42483 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.657193 525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 760dc8b205a441b0bf16670e8dd716a7)
I20260811 14:30:48.657205 526 raft_consensus.cc:482] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 760dc8b205a441b0bf16670e8dd716a7)
I20260811 14:30:48.657379 525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:48.657374 526 raft_consensus.cc:504] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:48.657694 525 leader_election.cc:288] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483), e431a103e3264c75ae234353c0158cc6 (127.23.186.252:43991)
I20260811 14:30:48.657975 526 leader_election.cc:288] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483), 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253:34957)
I20260811 14:30:48.658049 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.657831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55448)
0811 14:30:48.657957 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.658007 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
W20260811 14:30:48.658159 32359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.658304 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.657835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55440)
0811 14:30:48.658164 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.658263 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":79,"threads_started":1}
W20260811 14:30:48.658511 32125 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.658659 528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.658568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55456)
0811 14:30:48.658604 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.658630 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 14:30:48.658731 32359 leader_election.cc:334] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.659210 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.659004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55464)
0811 14:30:48.659063 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.659182 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 14:30:48.659386 32125 leader_election.cc:334] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.661245 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.657844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:34957 (local address 127.0.0.1:35966)
0811 14:30:48.658087 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.658097 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:48.658160 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:48.658445 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:48.658447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:48.658449 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:48.658513 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:48.658515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.659467 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.659468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:48.660348 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:48.660351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.660990 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.660992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:48.661049 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:48.661054 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:48.661118 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 14:30:48.661165 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":44,"threads_started":1}
I20260811 14:30:48.661280 532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.658165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.23.186.253:34957)
0811 14:30:48.658245 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:48.658247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:48.658258 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:48.658288 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:48.658289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:48.658300 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:48.658386 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:48.658601 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.659371 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.660428 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.660922 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.660979 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:48.660987 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:48.661145 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 14:30:48.661195 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":67,"thread_start_us":43,"threads_started":1}
I20260811 14:30:48.661531 530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.658006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.23.186.252:43991)
0811 14:30:48.658424 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:48.658426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:48.658434 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:48.658459 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:48.658460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:48.658471 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:48.658534 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:48.658771 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.659579 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.660580 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.661139 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.661199 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:48.661207 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:48.661426 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 14:30:48.661488 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":42,"threads_started":1}
I20260811 14:30:48.661518 32154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e431a103e3264c75ae234353c0158cc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "15528f761e2e46dca9dfd2c4be1d7808" is_pre_election: true
I20260811 14:30:48.661645 32154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e431a103e3264c75ae234353c0158cc6 in term 1.
I20260811 14:30:48.661758 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.658091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:43991 (local address 127.0.0.1:55396)
0811 14:30:48.658280 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.658293 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:48.658343 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:48.658584 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:48.658585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:48.658588 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:48.658685 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:48.658688 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.659674 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.659674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:48.660499 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:48.660501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:48.661199 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:48.661200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:48.661263 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:48.661268 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:48.661650 (+ 382us) client_negotiation.cc:240] Negotiation successful
0811 14:30:48.661697 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":34,"thread_start_us":48,"threads_started":1}
I20260811 14:30:48.661856 32357 leader_election.cc:302] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15528f761e2e46dca9dfd2c4be1d7808, e431a103e3264c75ae234353c0158cc6; no voters: 760dc8b205a441b0bf16670e8dd716a7
I20260811 14:30:48.661998 32389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15528f761e2e46dca9dfd2c4be1d7808" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e431a103e3264c75ae234353c0158cc6" is_pre_election: true
I20260811 14:30:48.662106 32389 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15528f761e2e46dca9dfd2c4be1d7808 in term 1.
I20260811 14:30:48.662186 526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 14:30:48.662235 526 raft_consensus.cc:482] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 760dc8b205a441b0bf16670e8dd716a7)
I20260811 14:30:48.662267 526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:30:48.662273 32126 leader_election.cc:302] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15528f761e2e46dca9dfd2c4be1d7808, e431a103e3264c75ae234353c0158cc6; no voters: 760dc8b205a441b0bf16670e8dd716a7
I20260811 14:30:48.662381 525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 14:30:48.662431 525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 760dc8b205a441b0bf16670e8dd716a7)
I20260811 14:30:48.662460 525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:30:48.663261 526 raft_consensus.cc:504] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:48.663298 525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:48.663420 526 leader_election.cc:288] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 2 election: Requested vote from peers 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483), 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253:34957)
I20260811 14:30:48.663414 525 leader_election.cc:288] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [CANDIDATE]: Term 2 election: Requested vote from peers 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483), e431a103e3264c75ae234353c0158cc6 (127.23.186.252:43991)
I20260811 14:30:48.663595 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.663495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55476)
0811 14:30:48.663533 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.663573 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260811 14:30:48.663740 32389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "15528f761e2e46dca9dfd2c4be1d7808" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e431a103e3264c75ae234353c0158cc6"
I20260811 14:30:48.663777 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.663495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55474)
0811 14:30:48.663723 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.663754 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260811 14:30:48.663779 32154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e431a103e3264c75ae234353c0158cc6" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "15528f761e2e46dca9dfd2c4be1d7808"
I20260811 14:30:48.663841 32389 raft_consensus.cc:2379] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 15528f761e2e46dca9dfd2c4be1d7808 in current term 2: Already voted for candidate e431a103e3264c75ae234353c0158cc6 in this term.
I20260811 14:30:48.663887 32154 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e431a103e3264c75ae234353c0158cc6 in current term 2: Already voted for candidate 15528f761e2e46dca9dfd2c4be1d7808 in this term.
I20260811 14:30:48.664039 527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.663899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55494)
0811 14:30:48.663982 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.664010 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 14:30:48.664110 32359 leader_election.cc:334] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.664183 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:48.664070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:55510)
0811 14:30:48.664125 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:48.664158 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:30:48.664276 32125 leader_election.cc:334] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:48.664285 32357 leader_election.cc:302] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e431a103e3264c75ae234353c0158cc6; no voters: 15528f761e2e46dca9dfd2c4be1d7808, 760dc8b205a441b0bf16670e8dd716a7
I20260811 14:30:48.664376 32125 leader_election.cc:302] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15528f761e2e46dca9dfd2c4be1d7808; no voters: 760dc8b205a441b0bf16670e8dd716a7, e431a103e3264c75ae234353c0158cc6
I20260811 14:30:48.664384 526 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 14:30:48.664445 525 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 14:30:49.153592 535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:49.153181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:52569)
0811 14:30:49.153377 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:49.153440 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":109,"threads_started":1}
I20260811 14:30:49.153921 536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:49.153462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:33343)
0811 14:30:49.153841 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:49.153878 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":68,"threads_started":1}
I20260811 14:30:49.154107 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:49.153729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:58949)
0811 14:30:49.153907 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:49.153955 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":62,"threads_started":1}
I20260811 14:30:50.154614 538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.154304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:51463)
0811 14:30:50.154494 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.154540 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260811 14:30:50.154654 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.154413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:52157)
0811 14:30:50.154548 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.154583 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":61,"threads_started":1}
I20260811 14:30:50.154891 540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.154646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:42425)
0811 14:30:50.154811 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.154849 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":97,"threads_started":1}
I20260811 14:30:50.759738 541 raft_consensus.cc:482] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:50.759856 541 raft_consensus.cc:504] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:50.760075 541 leader_election.cc:288] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483), 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253:34957)
I20260811 14:30:50.760329 32154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e431a103e3264c75ae234353c0158cc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "15528f761e2e46dca9dfd2c4be1d7808" is_pre_election: true
I20260811 14:30:50.760421 32154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e431a103e3264c75ae234353c0158cc6 in term 2.
I20260811 14:30:50.760443 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.760214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:59686)
0811 14:30:50.760340 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.760381 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260811 14:30:50.760598 32357 leader_election.cc:302] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15528f761e2e46dca9dfd2c4be1d7808, e431a103e3264c75ae234353c0158cc6; no voters:
I20260811 14:30:50.760871 541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 14:30:50.760911 541 raft_consensus.cc:482] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:50.760926 541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:30:50.760960 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.760838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:59692)
0811 14:30:50.760887 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.760932 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:30:50.761029 32359 leader_election.cc:334] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:50.761755 541 raft_consensus.cc:504] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:50.761909 541 leader_election.cc:288] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 3 election: Requested vote from peers 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483), 15528f761e2e46dca9dfd2c4be1d7808 (127.23.186.253:34957)
I20260811 14:30:50.762084 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.761995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:59706)
0811 14:30:50.762029 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.762054 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260811 14:30:50.762063 32154 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e431a103e3264c75ae234353c0158cc6" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "15528f761e2e46dca9dfd2c4be1d7808"
I20260811 14:30:50.762161 32154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:30:50.762382 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.762293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:59720)
0811 14:30:50.762339 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.762362 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:30:50.762463 32359 leader_election.cc:334] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
I20260811 14:30:50.762868 32154 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e431a103e3264c75ae234353c0158cc6 in term 3.
I20260811 14:30:50.763056 32357 leader_election.cc:302] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15528f761e2e46dca9dfd2c4be1d7808, e431a103e3264c75ae234353c0158cc6; no voters: 760dc8b205a441b0bf16670e8dd716a7
I20260811 14:30:50.763154 541 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 14:30:50.763263 541 raft_consensus.cc:686] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [term 3 LEADER]: Becoming Leader. State: Replica: e431a103e3264c75ae234353c0158cc6, State: Running, Role: LEADER
I20260811 14:30:50.763347 541 consensus_queue.cc:237] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } }
I20260811 14:30:50.763756 546 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e431a103e3264c75ae234353c0158cc6. Latest consensus state: current_term: 3 leader_uuid: "e431a103e3264c75ae234353c0158cc6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:50.763854 546 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:50.764118 548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:50.764528 548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:50.764657 548 catalog_manager.cc:1259] Loaded cluster ID: a1e47cbad435406387ae424fea8d7e30
I20260811 14:30:50.764722 548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:50.764930 548 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:50.765030 548 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Loaded TSK: 0
I20260811 14:30:50.765210 548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:30:50.852420 32372 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:50.854110 32372 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:50.855209 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.855090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:59728)
0811 14:30:50.855150 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.855182 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 14:30:50.855257 32154 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [term 3 FOLLOWER]: Refusing update from remote peer e431a103e3264c75ae234353c0158cc6: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 14:30:50.855522 541 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 14:30:50.855611 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.855494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.0.0.1:59736)
0811 14:30:50.855551 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.855578 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 14:30:50.855705 32359 consensus_peers.cc:487] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 -> Peer 760dc8b205a441b0bf16670e8dd716a7 (127.23.186.254:42483): Couldn't send request to peer 760dc8b205a441b0bf16670e8dd716a7. Status: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 14:30:50.856202 550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e431a103e3264c75ae234353c0158cc6. Latest consensus state: current_term: 3 leader_uuid: "e431a103e3264c75ae234353c0158cc6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:50.856328 550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:30:50.856825 546 sys_catalog.cc:455] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "e431a103e3264c75ae234353c0158cc6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:50.856923 546 sys_catalog.cc:458] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:50.857290 550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "e431a103e3264c75ae234353c0158cc6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "760dc8b205a441b0bf16670e8dd716a7" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42483 } } peers { permanent_uuid: "15528f761e2e46dca9dfd2c4be1d7808" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 34957 } attrs { promote: false } } peers { permanent_uuid: "e431a103e3264c75ae234353c0158cc6" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 43991 } attrs { promote: false } } }
I20260811 14:30:50.857378 550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15528f761e2e46dca9dfd2c4be1d7808 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:30:50.862663 542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.858898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:36245 (local address 127.0.0.1:58854)
0811 14:30:50.859115 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.859125 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:50.859207 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:50.859525 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:50.859526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:50.859528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:50.859594 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:50.859596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.860627 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.860628 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:50.861691 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:50.861694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.862385 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862387 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:50.862448 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:50.862454 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:30:50.862511 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:30:50.862567 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":35}
I20260811 14:30:50.862669 557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.858978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58854 (local address 127.23.186.193:36245)
0811 14:30:50.859095 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:50.859097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:50.859188 (+ 91us) server_negotiation.cc:407] Connection header received
0811 14:30:50.859277 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:50.859278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:50.859289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:50.859666 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:50.859713 (+ 47us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.860529 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.861778 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.862297 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:50.862384 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:50.862526 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 14:30:50.862572 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":72,"threads_started":1}
I20260811 14:30:50.862960 32510 tablet_service.cc:1467] Processing CreateTablet for tablet 381a7372734140009f0843db81086da2 (DEFAULT_TABLE table=test-table [id=2ab1adba1a7b4b23b3ead2b6a186a3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:50.863024 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.858985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:43197 (local address 127.0.0.1:41580)
0811 14:30:50.859478 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.859489 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:50.859558 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:50.859757 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:50.859759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:50.859761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:50.859814 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:50.859815 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.860598 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.860599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:50.861910 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:50.861912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.862782 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:50.862853 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:50.862858 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:50.862918 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:30:50.862956 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":56,"thread_start_us":89,"threads_started":1}
I20260811 14:30:50.863047 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.859376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41580 (local address 127.23.186.195:43197)
0811 14:30:50.859627 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:50.859630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:50.859638 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:50.859664 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:50.859665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:50.859675 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:50.859741 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:50.859879 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.860465 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862010 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.862448 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862523 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:50.862531 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:50.862934 (+ 403us) server_negotiation.cc:299] Negotiation successful
0811 14:30:50.862979 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":67,"threads_started":1}
I20260811 14:30:50.863137 32510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381a7372734140009f0843db81086da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:50.863549 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.858904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33035 (local address 127.0.0.1:53584)
0811 14:30:50.859494 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.859505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:50.859546 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:50.859806 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:50.859807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:50.859809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:50.859846 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:50.859847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.860796 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.860797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:50.861910 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:50.861912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.862664 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:50.863352 (+ 688us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:50.863357 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:50.863436 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:30:50.863478 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"mutex_wait_us":441,"thread_start_us":95,"threads_started":1}
I20260811 14:30:50.863567 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.859002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53584 (local address 127.23.186.194:33035)
0811 14:30:50.859644 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:50.859646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:50.859652 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:30:50.859685 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:50.859686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:50.859695 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:50.859798 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:50.859933 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.860682 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.861995 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:50.862584 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:50.862657 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:50.862665 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:50.863442 (+ 777us) server_negotiation.cc:299] Negotiation successful
0811 14:30:50.863490 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":621,"thread_start_us":68,"threads_started":1}
I20260811 14:30:50.864090 32672 tablet_service.cc:1467] Processing CreateTablet for tablet 381a7372734140009f0843db81086da2 (DEFAULT_TABLE table=test-table [id=2ab1adba1a7b4b23b3ead2b6a186a3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:50.864236 32672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381a7372734140009f0843db81086da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:50.864720 562 tablet_bootstrap.cc:492] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: Bootstrap starting.
I20260811 14:30:50.865152 562 tablet_bootstrap.cc:654] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:50.865768 565 tablet_bootstrap.cc:492] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Bootstrap starting.
I20260811 14:30:50.865784 562 tablet_bootstrap.cc:492] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: No bootstrap required, opened a new log
I20260811 14:30:50.865830 562 ts_tablet_manager.cc:1397] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:50.865818 368 tablet_service.cc:1467] Processing CreateTablet for tablet 381a7372734140009f0843db81086da2 (DEFAULT_TABLE table=test-table [id=2ab1adba1a7b4b23b3ead2b6a186a3f8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:50.865937 368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381a7372734140009f0843db81086da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:50.866019 562 raft_consensus.cc:348] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.866111 562 raft_consensus.cc:374] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:50.866144 562 raft_consensus.cc:729] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65c9aa5e5d004db183da0827fc6d75c5, State: Initialized, Role: FOLLOWER
I20260811 14:30:50.866212 565 tablet_bootstrap.cc:654] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:50.866214 562 consensus_queue.cc:260] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.866355 562 ts_tablet_manager.cc:1428] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:50.867046 565 tablet_bootstrap.cc:492] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: No bootstrap required, opened a new log
I20260811 14:30:50.867115 565 ts_tablet_manager.cc:1397] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:50.867252 565 raft_consensus.cc:348] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.867323 565 raft_consensus.cc:374] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:50.867355 565 raft_consensus.cc:729] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07b4360466d842848a1fb5033a4ccf22, State: Initialized, Role: FOLLOWER
I20260811 14:30:50.867374 568 tablet_bootstrap.cc:492] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Bootstrap starting.
I20260811 14:30:50.867483 565 consensus_queue.cc:260] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.867684 565 ts_tablet_manager.cc:1428] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:50.867761 568 tablet_bootstrap.cc:654] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:50.868178 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.867950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:46591)
0811 14:30:50.868077 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.868117 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":82,"threads_started":1}
I20260811 14:30:50.868314 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.868003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:59871)
0811 14:30:50.868236 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.868275 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260811 14:30:50.868602 32749 heartbeater.cc:503] Master 127.23.186.252:43991 was elected leader, sending a full tablet report...
I20260811 14:30:50.868602 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.868509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:52011)
0811 14:30:50.868548 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.868572 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 14:30:50.868968 568 tablet_bootstrap.cc:492] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: No bootstrap required, opened a new log
I20260811 14:30:50.869025 568 ts_tablet_manager.cc:1397] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:50.869195 568 raft_consensus.cc:348] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.869268 568 raft_consensus.cc:374] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:50.869282 568 raft_consensus.cc:729] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c599349c89f047c2975dbea16464dca1, State: Initialized, Role: FOLLOWER
I20260811 14:30:50.869338 568 consensus_queue.cc:260] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.869477 568 ts_tablet_manager.cc:1428] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:50.869865 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.869667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:53129)
0811 14:30:50.869795 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.869832 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":87,"threads_started":1}
I20260811 14:30:50.869923 446 heartbeater.cc:503] Master 127.23.186.252:43991 was elected leader, sending a full tablet report...
W20260811 14:30:50.949627 32592 tablet.cc:2369] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:50.974848 574 raft_consensus.cc:482] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:50.975003 574 raft_consensus.cc:504] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.975214 574 leader_election.cc:288] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035), 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:50.975479 32695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381a7372734140009f0843db81086da2" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b4360466d842848a1fb5033a4ccf22" is_pre_election: true
I20260811 14:30:50.975590 32695 raft_consensus.cc:2454] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 0.
I20260811 14:30:50.975513 32534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381a7372734140009f0843db81086da2" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9aa5e5d004db183da0827fc6d75c5" is_pre_election: true
I20260811 14:30:50.975663 32534 raft_consensus.cc:2454] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 0.
I20260811 14:30:50.975768 311 leader_election.cc:302] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07b4360466d842848a1fb5033a4ccf22, c599349c89f047c2975dbea16464dca1; no voters:
I20260811 14:30:50.975871 574 raft_consensus.cc:2781] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:50.975914 574 raft_consensus.cc:482] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:50.975934 574 raft_consensus.cc:3037] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:50.976747 574 raft_consensus.cc:504] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.976905 574 leader_election.cc:288] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 election: Requested vote from peers 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035), 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245)
I20260811 14:30:50.977080 32695 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381a7372734140009f0843db81086da2" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07b4360466d842848a1fb5033a4ccf22"
I20260811 14:30:50.977135 32534 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381a7372734140009f0843db81086da2" candidate_uuid: "c599349c89f047c2975dbea16464dca1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "65c9aa5e5d004db183da0827fc6d75c5"
I20260811 14:30:50.977201 32695 raft_consensus.cc:3037] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:50.977212 32534 raft_consensus.cc:3037] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:50.978029 32534 raft_consensus.cc:2454] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 1.
I20260811 14:30:50.978030 32695 raft_consensus.cc:2454] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c599349c89f047c2975dbea16464dca1 in term 1.
I20260811 14:30:50.978264 311 leader_election.cc:302] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 07b4360466d842848a1fb5033a4ccf22, 65c9aa5e5d004db183da0827fc6d75c5, c599349c89f047c2975dbea16464dca1; no voters:
I20260811 14:30:50.978380 574 raft_consensus.cc:2781] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:50.978566 574 raft_consensus.cc:686] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 1 LEADER]: Becoming Leader. State: Replica: c599349c89f047c2975dbea16464dca1, State: Running, Role: LEADER
I20260811 14:30:50.978638 574 consensus_queue.cc:237] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } }
I20260811 14:30:50.978978 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:50.978830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:42523)
0811 14:30:50.978880 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:50.978938 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260811 14:30:50.979518 32372 catalog_manager.cc:5495] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c599349c89f047c2975dbea16464dca1 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "c599349c89f047c2975dbea16464dca1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c599349c89f047c2975dbea16464dca1" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43197 } health_report { overall_health: HEALTHY } } }
I20260811 14:30:51.008584 561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.006239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41586 (local address 127.23.186.195:43197)
0811 14:30:51.006285 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:51.006288 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:51.006297 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:51.006330 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:51.006331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:51.006343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:51.006410 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:51.006549 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:51.007065 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:51.007845 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:51.008048 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:51.008133 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:51.008290 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:51.008333 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:51.008395 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:51.008460 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:30:51.008486 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:30:51.009722 32534 raft_consensus.cc:1261] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Refusing update from remote peer c599349c89f047c2975dbea16464dca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:51.009764 32695 raft_consensus.cc:1261] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Refusing update from remote peer c599349c89f047c2975dbea16464dca1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:51.009999 574 consensus_queue.cc:1035] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07b4360466d842848a1fb5033a4ccf22" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33035 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:51.010109 576 consensus_queue.cc:1035] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "65c9aa5e5d004db183da0827fc6d75c5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:51.010509 32588 heartbeater.cc:503] Master 127.23.186.252:43991 was elected leader, sending a full tablet report...
I20260811 14:30:51.010903 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.010646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:48499)
0811 14:30:51.010828 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:51.010870 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 14:30:51.011250 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.011115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:58667)
0811 14:30:51.011180 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:51.011216 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 14:30:51.011911 575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.011799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.195:41367)
0811 14:30:51.011844 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:51.011880 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 14:30:51.011926 573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.011826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.194:41079)
0811 14:30:51.011861 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:51.011891 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260811 14:30:51.012535 579 mvcc.cc:205] Tried to move back new op lower bound from 7317334634533572608 to 7317334634409541632. Current Snapshot: MvccSnapshot[applied={T|T < 7317334634533572608}]
I20260811 14:30:51.013221 572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.013087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42483 (local address 127.23.186.193:46289)
0811 14:30:51.013157 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:51.013190 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:42483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 14:30:51.014220 32376 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:51.014307 32376 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55390:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:51.015517 32376 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e431a103e3264c75ae234353c0158cc6: Sending DeleteTablet for 3 replicas of tablet 381a7372734140009f0843db81086da2
I20260811 14:30:51.015805 32672 tablet_service.cc:1514] Processing DeleteTablet for tablet 381a7372734140009f0843db81086da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:51 UTC) from {username='oryx', principal='oryx/127.23.186.252@KRBTEST.COM'} at 127.0.0.1:53584
I20260811 14:30:51.015882 32510 tablet_service.cc:1514] Processing DeleteTablet for tablet 381a7372734140009f0843db81086da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:51 UTC) from {username='oryx', principal='oryx/127.23.186.252@KRBTEST.COM'} at 127.0.0.1:58854
I20260811 14:30:51.016000 368 tablet_service.cc:1514] Processing DeleteTablet for tablet 381a7372734140009f0843db81086da2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:51 UTC) from {username='oryx', principal='oryx/127.23.186.252@KRBTEST.COM'} at 127.0.0.1:41580
I20260811 14:30:51.016083 589 tablet_replica.cc:330] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5: stopping tablet replica
I20260811 14:30:51.016137 589 raft_consensus.cc:2229] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:51.016201 589 raft_consensus.cc:2258] T 381a7372734140009f0843db81086da2 P 65c9aa5e5d004db183da0827fc6d75c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:51.016207 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 32336
I20260811 14:30:51.016222 590 tablet_replica.cc:330] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: stopping tablet replica
I20260811 14:30:51.016811 590 raft_consensus.cc:2229] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:51.016893 590 raft_consensus.cc:2258] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:51.016824 588 tablet_replica.cc:330] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: stopping tablet replica
I20260811 14:30:51.017037 588 raft_consensus.cc:2229] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:51.017143 588 raft_consensus.cc:2258] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:51.018020 590 ts_tablet_manager.cc:1905] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:51.018501 588 ts_tablet_manager.cc:1905] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:51.019742 590 ts_tablet_manager.cc:1918] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:51.019825 590 log.cc:1196] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/381a7372734140009f0843db81086da2
I20260811 14:30:51.020073 590 ts_tablet_manager.cc:1939] T 381a7372734140009f0843db81086da2 P c599349c89f047c2975dbea16464dca1: Deleting consensus metadata
I20260811 14:30:51.020447 32360 catalog_manager.cc:4841] TS c599349c89f047c2975dbea16464dca1 (127.23.186.195:43197): tablet 381a7372734140009f0843db81086da2 (table test-table [id=2ab1adba1a7b4b23b3ead2b6a186a3f8]) successfully deleted
I20260811 14:30:51.020802 588 ts_tablet_manager.cc:1918] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:51.020889 588 log.cc:1196] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/381a7372734140009f0843db81086da2
I20260811 14:30:51.021155 588 ts_tablet_manager.cc:1939] T 381a7372734140009f0843db81086da2 P 07b4360466d842848a1fb5033a4ccf22: Deleting consensus metadata
I20260811 14:30:51.021469 32359 catalog_manager.cc:4841] TS 07b4360466d842848a1fb5033a4ccf22 (127.23.186.194:33035): tablet 381a7372734140009f0843db81086da2 (table test-table [id=2ab1adba1a7b4b23b3ead2b6a186a3f8]) successfully deleted
I20260811 14:30:51.032032 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:51.031856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:36245 (local address 127.0.0.1:58860)
0811 14:30:51.031936 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:51.031978 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:36245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 14:30:51.032258 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 32597
W20260811 14:30:51.032294 32362 catalog_manager.cc:4584] TS 65c9aa5e5d004db183da0827fc6d75c5 (127.23.186.193:36245): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 381a7372734140009f0843db81086da2: Network error: Client connection negotiation failed: client connection to 127.23.186.193:36245: connect: Connection refused (error 111)
I20260811 14:30:51.041013 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 32758
I20260811 14:30:51.049520 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 31979
I20260811 14:30:51.057565 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 32205
2026-08-11T14:30:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7047 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:51 dist-test-slave-cbsf krb5kdc[594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:51 dist-test-slave-cbsf krb5kdc[594](info): set up 2 sockets
Aug 11 14:30:51 dist-test-slave-cbsf krb5kdc[594](info): commencing operation
krb5kdc: starting...
W20260811 14:30:52.190634 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab.
Entry for principal oryx/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab.
I20260811 14:30:52.320647 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:42663
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab
--principal=oryx/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:42663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:52.405944 608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:52.406232 608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:52.406275 608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:52.407647 608 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:52.407714 608 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 14:30:52.407737 608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:52.407768 608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:52.407799 608 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:52.407830 608 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:52.409557 608 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:42663
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.254.keytab
--principal=oryx/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:42663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:52.409866 608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:52.410120 608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:52.415205 614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:52.415148 618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:52.415508 608 server_base.cc:1034] running on GCE node
W20260811 14:30:52.415153 613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:52.415863 608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:52.416229 608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:52.417394 608 hybrid_clock.cc:648] HybridClock initialized: now 1786458652417390 us; error 41 us; skew 500 ppm
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:52.420001 608 init.cc:377] Logged in from keytab as oryx/127.23.186.254@KRBTEST.COM (short username oryx)
I20260811 14:30:52.420468 608 webserver.cc:460] Webserver started at http://127.23.186.254:39551/ using document root <none> and password file <none>
I20260811 14:30:52.420766 608 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:52.420840 608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:52.421003 608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:52.422197 608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "c4e44cf3244a48a99d40404370f7c604"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "72df634d5b6d7add56194d36a4ecfe87"
server_key_iv: "392c59d941769833f5a1c2d949908178"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.422580 608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "c4e44cf3244a48a99d40404370f7c604"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "72df634d5b6d7add56194d36a4ecfe87"
server_key_iv: "392c59d941769833f5a1c2d949908178"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.424012 608 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:30:52.424825 626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.425014 608 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:52.425123 608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "c4e44cf3244a48a99d40404370f7c604"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "72df634d5b6d7add56194d36a4ecfe87"
server_key_iv: "392c59d941769833f5a1c2d949908178"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.425207 608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:52.440553 608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:52.442589 608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:52.442767 608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:52.447734 608 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:42663
I20260811 14:30:52.447786 702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:42663 every 8 connection(s)
I20260811 14:30:52.448134 608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:52.448834 703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:52.451241 703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Bootstrap starting.
I20260811 14:30:52.451902 703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:52.452210 703 log.cc:824] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:52.453033 703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: No bootstrap required, opened a new log
I20260811 14:30:52.455209 703 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e44cf3244a48a99d40404370f7c604" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42663 } }
I20260811 14:30:52.455387 703 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:52.455431 703 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4e44cf3244a48a99d40404370f7c604, State: Initialized, Role: FOLLOWER
I20260811 14:30:52.455636 703 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e44cf3244a48a99d40404370f7c604" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42663 } }
I20260811 14:30:52.455749 703 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:52.455802 703 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:52.455861 703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:52.456465 703 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e44cf3244a48a99d40404370f7c604" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42663 } }
I20260811 14:30:52.456596 703 leader_election.cc:302] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4e44cf3244a48a99d40404370f7c604; no voters:
I20260811 14:30:52.456836 703 leader_election.cc:288] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:52.456931 706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:52.456951 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 608
I20260811 14:30:52.457053 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:52.457113 706 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [term 1 LEADER]: Becoming Leader. State: Replica: c4e44cf3244a48a99d40404370f7c604, State: Running, Role: LEADER
I20260811 14:30:52.457193 703 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:52.457231 706 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e44cf3244a48a99d40404370f7c604" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42663 } }
I20260811 14:30:52.457293 24299 external_mini_cluster.cc:1428] Setting key 58f54967714750f77c33671c8ec6d4ad
I20260811 14:30:52.457587 707 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4e44cf3244a48a99d40404370f7c604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e44cf3244a48a99d40404370f7c604" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42663 } } }
I20260811 14:30:52.457690 707 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:52.457929 709 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4e44cf3244a48a99d40404370f7c604. Latest consensus state: current_term: 1 leader_uuid: "c4e44cf3244a48a99d40404370f7c604" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4e44cf3244a48a99d40404370f7c604" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42663 } } }
I20260811 14:30:52.458079 709 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604 [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:52.458225 712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:52.458783 712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:52.460489 712 catalog_manager.cc:1347] Generated new cluster ID: 22057b0782c949878952b820e98551e8
I20260811 14:30:52.460542 712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:52.470448 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.457828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.23.186.254:42663)
0811 14:30:52.458025 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.458036 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.458908 (+ 872us) server_negotiation.cc:407] Connection header received
0811 14:30:52.459219 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.459230 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.459299 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.459457 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:52.460238 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.461675 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.462757 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.463443 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.463565 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.466568 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:52.466573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:52.466573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:52.466588 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:52.468633 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.469088 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.469091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.469091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.469214 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.469433 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.469434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.469434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.469770 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:52.469923 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:52.470060 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 14:30:52.470213 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":83,"threads_started":1}
I20260811 14:30:52.483551 712 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:52.484333 712 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:52.490432 712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Generated new TSK 0
I20260811 14:30:52.490624 712 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab.
Entry for principal oryx/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab.
I20260811 14:30:52.520581 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42663
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:52.606204 730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:52.606426 730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:52.606455 730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:52.607695 730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:52.607782 730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:52.607822 730 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:52.607852 730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:52.609525 730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:42663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:52.609840 730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:52.610108 730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:52.615736 736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:52.615768 730 server_base.cc:1034] running on GCE node
W20260811 14:30:52.615736 737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:52.615736 739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:52.616211 730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:52.616501 730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:52.617683 730 hybrid_clock.cc:648] HybridClock initialized: now 1786458652617656 us; error 63 us; skew 500 ppm
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:52.620311 730 init.cc:377] Logged in from keytab as oryx/127.23.186.193@KRBTEST.COM (short username oryx)
I20260811 14:30:52.620918 730 webserver.cc:460] Webserver started at http://127.23.186.193:44877/ using document root <none> and password file <none>
I20260811 14:30:52.621191 730 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:52.621277 730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:52.621438 730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:52.622557 730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "4109e6aee02d4833946012e3ec4d0b91"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "c0e07677c5ef62614990643a529a1ee1"
server_key_iv: "139eb6caa516398b0f7f8e8bf290d200"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.622951 730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "4109e6aee02d4833946012e3ec4d0b91"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "c0e07677c5ef62614990643a529a1ee1"
server_key_iv: "139eb6caa516398b0f7f8e8bf290d200"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.624433 730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:52.625586 748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.625777 730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:52.625885 730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "4109e6aee02d4833946012e3ec4d0b91"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "c0e07677c5ef62614990643a529a1ee1"
server_key_iv: "139eb6caa516398b0f7f8e8bf290d200"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.625973 730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:52.635412 730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:52.637380 730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:52.637548 730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:52.638145 730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:52.638211 730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.638252 730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:52.638276 730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:52.646047 730 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:37621
I20260811 14:30:52.646088 880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:37621 every 8 connection(s)
I20260811 14:30:52.646517 730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:52.649413 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.638585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:37005 (local address 127.23.186.254:42663)
0811 14:30:52.638661 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.638663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.638898 (+ 235us) server_negotiation.cc:407] Connection header received
0811 14:30:52.639696 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.639698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.639712 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.639787 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:52.640744 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.641280 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.643021 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.643258 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.643348 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.646319 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:52.646320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:52.646321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:52.646334 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:52.647903 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.648356 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.648357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.648358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.648428 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.648704 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.648706 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.648706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.648835 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:52.648910 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:52.649216 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 14:30:52.649320 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:30:52.649581 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.638553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.23.186.193:37005)
0811 14:30:52.638737 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:52.638790 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:52.639521 (+ 731us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:52.639868 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:52.639873 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:52.640283 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:52.640614 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.640621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.641388 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.641390 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.642916 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.642920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.643371 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.643373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.643552 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.644106 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:52.644116 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:52.646183 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:52.648015 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.648023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.648026 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.648269 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.648500 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.648501 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.648502 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.648619 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.648913 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:52.648921 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:52.649074 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 14:30:52.649179 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 14:30:52.649347 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":99,"threads_started":1}
I20260811 14:30:52.650079 881 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42663
I20260811 14:30:52.650168 881 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:52.650372 881 heartbeater.cc:511] Master 127.23.186.254:42663 requested a full tablet report, sending...
I20260811 14:30:52.650938 646 ts_manager.cc:194] Registered new tserver with Master: 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621)
I20260811 14:30:52.651639 646 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:37005
I20260811 14:30:52.656507 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.651174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:44733 (local address 127.23.186.254:42663)
0811 14:30:52.651217 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.651220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.651339 (+ 119us) server_negotiation.cc:407] Connection header received
0811 14:30:52.651413 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.651414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.651424 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.651480 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:52.651808 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.652355 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.653126 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.653337 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.653422 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.654269 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:52.654271 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:52.654271 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:52.654278 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:52.655369 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.655719 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.655721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.655721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.655780 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.655982 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.655983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.655983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.656098 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:52.656176 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:52.656338 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 14:30:52.656418 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:30:52.656522 888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.651120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.23.186.193:44733)
0811 14:30:52.651263 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:52.651273 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:52.651354 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:52.651485 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:52.651485 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:52.651661 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:52.651722 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.651724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.652455 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.652456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.653041 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.653043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.653424 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.653425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.653498 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.653884 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:52.653888 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:52.654180 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:52.655454 (+ 1274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.655455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.655455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.655644 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.655857 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.655857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.655858 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.655918 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.656168 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:52.656169 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:52.656227 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 14:30:52.656312 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 14:30:52.656419 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260811 14:30:52.656670 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 730
I20260811 14:30:52.656810 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:52.657018 24299 external_mini_cluster.cc:1428] Setting key eaca5c5defc5484b63ba4e1078b034cb
WARNING: no policy specified for oryx/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab.
Entry for principal oryx/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab.
I20260811 14:30:52.683609 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42663
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:52.769493 891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:52.769709 891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:52.769735 891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:52.770856 891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:52.770892 891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:52.770913 891 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:52.770931 891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:52.772419 891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:42663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:52.772627 891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:52.772895 891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:52.778198 900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:52.778191 896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:52.778191 897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:52.778332 891 server_base.cc:1034] running on GCE node
I20260811 14:30:52.778689 891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:52.778949 891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:52.780092 891 hybrid_clock.cc:648] HybridClock initialized: now 1786458652780071 us; error 36 us; skew 500 ppm
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:52.782776 891 init.cc:377] Logged in from keytab as oryx/127.23.186.194@KRBTEST.COM (short username oryx)
I20260811 14:30:52.783267 891 webserver.cc:460] Webserver started at http://127.23.186.194:34345/ using document root <none> and password file <none>
I20260811 14:30:52.783492 891 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:52.783536 891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:52.783640 891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:52.784664 891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "864cffd627ff4fc49b4387a8c03865b9"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "1dc3750c1a7d150a7c838a2f7f180d0d"
server_key_iv: "26ad204a27bf90ac3250776a46a26f5d"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.785043 891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "864cffd627ff4fc49b4387a8c03865b9"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "1dc3750c1a7d150a7c838a2f7f180d0d"
server_key_iv: "26ad204a27bf90ac3250776a46a26f5d"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.786350 891 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:52.787075 908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.787235 891 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:52.787330 891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "864cffd627ff4fc49b4387a8c03865b9"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "1dc3750c1a7d150a7c838a2f7f180d0d"
server_key_iv: "26ad204a27bf90ac3250776a46a26f5d"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.787386 891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:52.805193 891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:52.807369 891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:52.807494 891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:52.807977 891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:52.808013 891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.808036 891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:52.808054 891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.815395 891 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:43013
I20260811 14:30:52.815450 1041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:43013 every 8 connection(s)
I20260811 14:30:52.815788 891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:52.818925 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.808622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:59017 (local address 127.23.186.254:42663)
0811 14:30:52.808674 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.808676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.808947 (+ 271us) server_negotiation.cc:407] Connection header received
0811 14:30:52.809549 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.809551 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.809564 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.809627 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:52.810610 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.811438 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.813241 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.813477 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.813538 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.816398 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:52.816400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:52.816400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:52.816408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:52.817597 (+ 1189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.817994 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.817994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.817995 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.818047 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.818318 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.818319 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.818319 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.818446 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:52.818499 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:52.818789 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 14:30:52.818859 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:30:52.819201 917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.808622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.23.186.194:59017)
0811 14:30:52.808819 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:52.808856 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:52.809436 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:52.809751 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:52.809756 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:52.810179 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:52.810494 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.810500 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.811554 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.811555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.813138 (+ 1583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.813142 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.813586 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.813588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.813762 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.814417 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:52.814427 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:52.816302 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:52.817673 (+ 1371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.817680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.817682 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.817917 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.818132 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.818133 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.818134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.818248 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.818518 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:52.818524 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:52.818665 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 14:30:52.818759 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 14:30:52.818942 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260811 14:30:52.819824 1042 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42663
I20260811 14:30:52.819949 1042 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:52.819979 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 891
I20260811 14:30:52.820120 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:52.820188 1042 heartbeater.cc:511] Master 127.23.186.254:42663 requested a full tablet report, sending...
I20260811 14:30:52.820339 24299 external_mini_cluster.cc:1428] Setting key 37e95f2630573f2056a9a00555322727
I20260811 14:30:52.820616 646 ts_manager.cc:194] Registered new tserver with Master: 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013)
I20260811 14:30:52.821134 646 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:59017
I20260811 14:30:52.828143 1048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.821072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.23.186.194:40933)
0811 14:30:52.821496 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:52.821508 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:52.821593 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:52.821770 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:52.821771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:52.822003 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:52.822076 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.822078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.822853 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.822854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.823387 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.823389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.823866 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.823867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.823916 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.824404 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:52.824409 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:52.824819 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:52.826909 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.826910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.826911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.827098 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.827321 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.827321 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.827322 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.827379 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.827815 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:52.827816 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:52.827873 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 14:30:52.827928 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:30:52.828040 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":370,"threads_started":1}
I20260811 14:30:52.828291 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.821283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:40933 (local address 127.23.186.254:42663)
0811 14:30:52.821326 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.821329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.821564 (+ 235us) server_negotiation.cc:407] Connection header received
0811 14:30:52.821665 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.821666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.821676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.821705 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:52.822183 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.822730 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.823476 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.823786 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.823863 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.824914 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:52.824916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:52.824917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:52.824925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:52.826812 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.827178 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.827180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.827181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.827249 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.827479 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.827480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.827480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.827743 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:52.827825 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:52.827985 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 14:30:52.828184 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
WARNING: no policy specified for oryx/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab.
Entry for principal oryx/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab.
I20260811 14:30:52.851559 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:42663
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:52.938521 1051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:52.938758 1051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:52.938786 1051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:52.940002 1051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:52.940049 1051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:52.940070 1051 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 14:30:52.940132 1051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:52.941783 1051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.23.186.254:42663
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:52.942061 1051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:52.942305 1051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:52.947451 1056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:52.947422 1060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:52.947433 1057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:52.947834 1051 server_base.cc:1034] running on GCE node
I20260811 14:30:52.948086 1051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:52.948338 1051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:52.949504 1051 hybrid_clock.cc:648] HybridClock initialized: now 1786458652949499 us; error 40 us; skew 500 ppm
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:52.952225 1051 init.cc:377] Logged in from keytab as oryx/127.23.186.195@KRBTEST.COM (short username oryx)
I20260811 14:30:52.952641 1051 webserver.cc:460] Webserver started at http://127.23.186.195:38505/ using document root <none> and password file <none>
I20260811 14:30:52.952931 1051 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:52.952982 1051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:52.953083 1051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:52.954154 1051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "0317d40b7c6344558d4bb6aac23fa59b"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "074841b8cb1e9067156772d53d61cd32"
server_key_iv: "1febc448d3572f73d4dd9ce17a4abdf2"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.954460 1051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "0317d40b7c6344558d4bb6aac23fa59b"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "074841b8cb1e9067156772d53d61cd32"
server_key_iv: "1febc448d3572f73d4dd9ce17a4abdf2"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.955817 1051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:52.956549 1069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.956766 1051 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:52.956866 1051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "0317d40b7c6344558d4bb6aac23fa59b"
format_stamp: "Formatted at 2026-08-11 14:30:52 on dist-test-slave-cbsf"
server_key: "074841b8cb1e9067156772d53d61cd32"
server_key_iv: "1febc448d3572f73d4dd9ce17a4abdf2"
server_key_version: "encryptionkey@0"
I20260811 14:30:52.956928 1051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:52.980419 1051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:52.982764 1051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:52.982897 1051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:52.983435 1051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:52.983536 1051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.983587 1051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:52.983626 1051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:52.991335 1051 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:36499
I20260811 14:30:52.991374 1208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:36499 every 8 connection(s)
Aug 11 14:30:52 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:52.991910 1051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:52.996717 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.984050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:59137 (local address 127.23.186.254:42663)
0811 14:30:52.984138 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.984142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.984304 (+ 162us) server_negotiation.cc:407] Connection header received
0811 14:30:52.984897 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.984899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.984914 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.984999 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:52.985949 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.986818 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.988587 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.988971 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.989054 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.992670 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:52.992672 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:52.992673 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:52.992684 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:52.994663 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.995331 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.995333 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.995333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.995409 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:52.995763 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:52.995765 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:52.995765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:52.995963 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:52.996042 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:52.996489 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 14:30:52.996596 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:30:52.996978 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.984000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.23.186.195:59137)
0811 14:30:52.984164 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:52.984208 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:52.984788 (+ 580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:52.985120 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:52.985124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:52.985506 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:52.985835 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.985841 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.986923 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.986924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.988478 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:52.988481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:52.989079 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:52.989081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:52.989263 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:52.989964 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:52.989978 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:52.992514 (+ 2536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:52.994786 (+ 2272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.994792 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.994796 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.995219 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.995494 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:52.995495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:52.995495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:52.995685 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:52.996084 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:52.996092 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:52.996297 (+ 205us) client_negotiation.cc:769] Sending connection context
0811 14:30:52.996453 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 14:30:52.996679 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260811 14:30:52.997601 1209 heartbeater.cc:347] Connected to a master server at 127.23.186.254:42663
I20260811 14:30:52.997743 1209 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:52.998023 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 1051
I20260811 14:30:52.998138 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:52.998363 24299 external_mini_cluster.cc:1428] Setting key 2d626b92e134ba4d3f4d58ff174be718
I20260811 14:30:52.999280 1209 heartbeater.cc:511] Master 127.23.186.254:42663 requested a full tablet report, sending...
I20260811 14:30:52.999688 646 ts_manager.cc:194] Registered new tserver with Master: 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:53.000514 646 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:59137
I20260811 14:30:53.005542 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.999174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:40323 (local address 127.23.186.254:42663)
0811 14:30:52.999219 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:52.999221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:52.999557 (+ 336us) server_negotiation.cc:407] Connection header received
0811 14:30:52.999641 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:52.999642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:52.999653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:52.999733 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:53.000142 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.000992 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.002002 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.002229 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.002292 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.003325 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:53.003326 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:53.003327 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:53.003333 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:53.004525 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.004938 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.004938 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.004939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.004991 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.005138 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.005138 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.005138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.005233 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:53.005291 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.005407 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.005464 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:53.005549 1215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:52.999255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.23.186.195:40323)
0811 14:30:52.999481 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:52.999492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:52.999571 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:52.999738 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:52.999739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:52.999970 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:53.000040 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.000048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.001126 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.001127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.001915 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.001918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.002304 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.002305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.002357 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.002873 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:53.002879 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:53.003222 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:53.004618 (+ 1396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:53.004619 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:53.004619 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:53.004878 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:53.005044 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:53.005044 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:53.005045 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:53.005095 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:53.005287 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:53.005288 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:53.005329 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.005392 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.005480 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
I20260811 14:30:53.010203 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458653, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:30:53.034636 716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:53.028198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41944 (local address 127.23.186.254:42663)
0811 14:30:53.028257 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.028259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.028377 (+ 118us) server_negotiation.cc:407] Connection header received
0811 14:30:53.028471 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.028472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.028492 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.028574 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:53.029450 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.029978 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.030814 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.031032 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.031100 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.034534 (+ 3434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41944: BlockingRecv error: recv got EOF from 127.0.0.1:41944 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458653, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:53.043011 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.035449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41950 (local address 127.23.186.254:42663)
0811 14:30:53.035516 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.035518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.035595 (+ 77us) server_negotiation.cc:407] Connection header received
0811 14:30:53.035654 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.035655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.035663 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.035715 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:53.036183 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.036680 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.037446 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.037631 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.037698 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.039926 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:53.039928 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:53.039928 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:53.039936 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:53.041418 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.041878 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.041879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.041880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.041971 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.042244 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.042245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.042245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.042407 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:53.042519 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.042757 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.042887 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:30:53.045295 646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:53.047232 646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:53.054597 1234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.049915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43013 (local address 127.0.0.1:34594)
0811 14:30:53.050088 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.050133 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.050204 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.051213 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.051214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.051217 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.051263 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.051265 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.052169 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.052170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.053414 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.053417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.054322 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.054323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.054400 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.054412 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.054467 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.054510 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":66,"thread_start_us":59,"threads_started":1}
I20260811 14:30:53.054683 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.050031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.23.186.194:43013)
0811 14:30:53.050475 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.050479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.050491 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:53.051104 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.051107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.051122 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.051214 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.051349 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.052065 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.053516 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.054221 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.054325 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.054403 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.054508 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.054552 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":94,"threads_started":1}
I20260811 14:30:53.054927 1238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.049941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36499 (local address 127.0.0.1:55690)
0811 14:30:53.050226 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.050238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.050303 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.050802 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.050806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.050809 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.050917 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.050919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.052612 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.052613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.053750 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.053753 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.054713 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.054714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.054782 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.054787 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.054831 (+ 44us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.054872 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":147,"threads_started":1}
I20260811 14:30:53.055742 960 tablet_service.cc:1467] Processing CreateTablet for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a (DEFAULT_TABLE table=test-table [id=0fbf260ca8da4103a422556d8f9eef8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.056012 1236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.049946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:37621 (local address 127.0.0.1:43104)
0811 14:30:53.050580 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.050591 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.050659 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.051292 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.051293 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.051295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.051332 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.051333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.052493 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.052494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.053759 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.053760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.055772 (+ 2012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.055773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.055837 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.055842 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.055898 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.055945 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":135,"threads_started":1}
I20260811 14:30:53.056099 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.050256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55690 (local address 127.23.186.195:36499)
0811 14:30:53.050475 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.050499 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.050510 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:53.050555 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.050558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.050575 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.050683 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.051016 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.052050 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.053856 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.054605 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.055842 (+ 1237us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.055895 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.055920 (+ 25us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.055984 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":104,"threads_started":1}
I20260811 14:30:53.056213 960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.056273 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.050243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43104 (local address 127.23.186.193:37621)
0811 14:30:53.050409 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.050414 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.050735 (+ 321us) server_negotiation.cc:407] Connection header received
0811 14:30:53.050878 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.050881 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.050901 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.050992 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.051432 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.052409 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.054893 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.055664 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.055754 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.055793 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.056124 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.056172 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260811 14:30:53.057165 1129 tablet_service.cc:1467] Processing CreateTablet for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a (DEFAULT_TABLE table=test-table [id=0fbf260ca8da4103a422556d8f9eef8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.057587 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.060842 1243 tablet_bootstrap.cc:492] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Bootstrap starting.
I20260811 14:30:53.061542 1243 tablet_bootstrap.cc:654] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.061609 809 tablet_service.cc:1467] Processing CreateTablet for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a (DEFAULT_TABLE table=test-table [id=0fbf260ca8da4103a422556d8f9eef8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.061981 809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.061973 1243 log.cc:824] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:53.062345 1244 tablet_bootstrap.cc:492] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Bootstrap starting.
I20260811 14:30:53.062997 1244 tablet_bootstrap.cc:654] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.063313 1244 log.cc:824] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:53.064209 1244 tablet_bootstrap.cc:492] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: No bootstrap required, opened a new log
I20260811 14:30:53.064303 1244 ts_tablet_manager.cc:1397] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:53.065073 1243 tablet_bootstrap.cc:492] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: No bootstrap required, opened a new log
I20260811 14:30:53.065176 1243 ts_tablet_manager.cc:1397] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 14:30:53.066347 1248 tablet_bootstrap.cc:492] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Bootstrap starting.
I20260811 14:30:53.066700 1244 raft_consensus.cc:348] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.066881 1244 raft_consensus.cc:374] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.066927 1244 raft_consensus.cc:729] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.067077 1248 tablet_bootstrap.cc:654] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.067061 1244 consensus_queue.cc:260] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.067363 1209 heartbeater.cc:503] Master 127.23.186.254:42663 was elected leader, sending a full tablet report...
I20260811 14:30:53.067309 1243 raft_consensus.cc:348] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.067482 1243 raft_consensus.cc:374] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.067463 1248 log.cc:824] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:53.067530 1243 raft_consensus.cc:729] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.067699 1244 ts_tablet_manager.cc:1428] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:53.067674 1243 consensus_queue.cc:260] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.067968 1243 ts_tablet_manager.cc:1428] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:30:53.068010 1042 heartbeater.cc:503] Master 127.23.186.254:42663 was elected leader, sending a full tablet report...
I20260811 14:30:53.068461 1248 tablet_bootstrap.cc:492] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: No bootstrap required, opened a new log
I20260811 14:30:53.068547 1248 ts_tablet_manager.cc:1397] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:30:53.070204 1248 raft_consensus.cc:348] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.070341 1248 raft_consensus.cc:374] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.070366 1248 raft_consensus.cc:729] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109e6aee02d4833946012e3ec4d0b91, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.070492 1248 consensus_queue.cc:260] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.070783 1248 ts_tablet_manager.cc:1428] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:30:53.070873 881 heartbeater.cc:503] Master 127.23.186.254:42663 was elected leader, sending a full tablet report...
I20260811 14:30:53.109952 1253 raft_consensus.cc:482] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:53.110153 1253 raft_consensus.cc:504] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.110548 1253 leader_election.cc:288] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013), 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:53.114704 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.110872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:59275 (local address 127.23.186.195:36499)
0811 14:30:53.110944 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.110947 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.110960 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:30:53.111045 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.111046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.111063 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.111207 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.111293 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.112091 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.113712 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.114296 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.114377 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.114386 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.114584 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.114627 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:30:53.114704 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.110888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:47523 (local address 127.23.186.194:43013)
0811 14:30:53.110947 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.110949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.111131 (+ 182us) server_negotiation.cc:407] Connection header received
0811 14:30:53.111239 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.111241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.111254 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.111329 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.111476 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.112151 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.113712 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.114293 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.114377 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.114386 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.114584 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.114627 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:30:53.114720 759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.110780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36499 (local address 127.23.186.193:59275)
0811 14:30:53.110863 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.110878 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.110943 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.111174 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.111175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.111177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.111241 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.111243 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.112174 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.112175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.113591 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.113596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.114390 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.114392 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.114490 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.114495 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.114557 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.114608 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":70}
I20260811 14:30:53.114718 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.110782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43013 (local address 127.23.186.193:47523)
0811 14:30:53.111022 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.111032 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.111089 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.111337 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.111338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.111339 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.111405 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.111406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.112226 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.112226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.113591 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.113596 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.114391 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.114392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.114501 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.114504 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.114557 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.114608 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":53,"thread_start_us":134,"threads_started":1}
I20260811 14:30:53.115206 1150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32fdcaaeceb44d5bbb8cebf7d3965f3a" candidate_uuid: "4109e6aee02d4833946012e3ec4d0b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317d40b7c6344558d4bb6aac23fa59b" is_pre_election: true
I20260811 14:30:53.115399 1150 raft_consensus.cc:2454] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4109e6aee02d4833946012e3ec4d0b91 in term 0.
I20260811 14:30:53.115435 992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32fdcaaeceb44d5bbb8cebf7d3965f3a" candidate_uuid: "4109e6aee02d4833946012e3ec4d0b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864cffd627ff4fc49b4387a8c03865b9" is_pre_election: true
I20260811 14:30:53.115599 992 raft_consensus.cc:2454] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4109e6aee02d4833946012e3ec4d0b91 in term 0.
I20260811 14:30:53.115679 750 leader_election.cc:302] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0317d40b7c6344558d4bb6aac23fa59b, 4109e6aee02d4833946012e3ec4d0b91; no voters:
I20260811 14:30:53.115854 1253 raft_consensus.cc:2781] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:53.115952 1253 raft_consensus.cc:482] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:53.115988 1253 raft_consensus.cc:3037] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.117223 1253 raft_consensus.cc:504] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.117396 1253 leader_election.cc:288] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [CANDIDATE]: Term 1 election: Requested vote from peers 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013), 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:53.117610 1150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32fdcaaeceb44d5bbb8cebf7d3965f3a" candidate_uuid: "4109e6aee02d4833946012e3ec4d0b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317d40b7c6344558d4bb6aac23fa59b"
I20260811 14:30:53.117610 992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32fdcaaeceb44d5bbb8cebf7d3965f3a" candidate_uuid: "4109e6aee02d4833946012e3ec4d0b91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864cffd627ff4fc49b4387a8c03865b9"
I20260811 14:30:53.117709 992 raft_consensus.cc:3037] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.117709 1150 raft_consensus.cc:3037] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.118669 1150 raft_consensus.cc:2454] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4109e6aee02d4833946012e3ec4d0b91 in term 1.
I20260811 14:30:53.118669 992 raft_consensus.cc:2454] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4109e6aee02d4833946012e3ec4d0b91 in term 1.
I20260811 14:30:53.118896 753 leader_election.cc:302] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109e6aee02d4833946012e3ec4d0b91, 864cffd627ff4fc49b4387a8c03865b9; no voters:
I20260811 14:30:53.119036 1253 raft_consensus.cc:2781] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:53.119231 1253 raft_consensus.cc:686] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 1 LEADER]: Becoming Leader. State: Replica: 4109e6aee02d4833946012e3ec4d0b91, State: Running, Role: LEADER
I20260811 14:30:53.119365 1253 consensus_queue.cc:237] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } }
I20260811 14:30:53.120365 646 catalog_manager.cc:5495] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193). New cstate: current_term: 1 leader_uuid: "4109e6aee02d4833946012e3ec4d0b91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } health_report { overall_health: HEALTHY } } }
I20260811 14:30:53.129136 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.126502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43108 (local address 127.23.186.193:37621)
0811 14:30:53.126568 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.126570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.126581 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:53.126625 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.126625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.126644 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.126715 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:53.126834 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.127303 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.128083 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.128275 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.128340 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.128598 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.128818 (+ 220us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.128886 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.128978 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.129009 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:30:53.131187 1150 raft_consensus.cc:1261] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Refusing update from remote peer 4109e6aee02d4833946012e3ec4d0b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.131187 992 raft_consensus.cc:1261] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Refusing update from remote peer 4109e6aee02d4833946012e3ec4d0b91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.131563 1253 consensus_queue.cc:1035] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.131748 1255 consensus_queue.cc:1035] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.134474 1258 mvcc.cc:205] Tried to move back new op lower bound from 7317334643222114304 to 7317334643177787392. Current Snapshot: MvccSnapshot[applied={T|T < 7317334643222114304}]
I20260811 14:30:53.134788 1261 mvcc.cc:205] Tried to move back new op lower bound from 7317334643222114304 to 7317334643177787392. Current Snapshot: MvccSnapshot[applied={T|T < 7317334643222114304}]
I20260811 14:30:53.135365 1259 mvcc.cc:205] Tried to move back new op lower bound from 7317334643222114304 to 7317334643177787392. Current Snapshot: MvccSnapshot[applied={T|T < 7317334643222114304 or (T in {7317334643222114304})}]
I20260811 14:30:53.137591 648 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:53.137693 648 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:53.138545 648 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Sending DeleteTablet for 3 replicas of tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a
I20260811 14:30:53.138911 960 tablet_service.cc:1514] Processing DeleteTablet for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:34594
I20260811 14:30:53.138897 1129 tablet_service.cc:1514] Processing DeleteTablet for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:55690
I20260811 14:30:53.138912 809 tablet_service.cc:1514] Processing DeleteTablet for tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43104
I20260811 14:30:53.139216 1273 tablet_replica.cc:330] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: stopping tablet replica
I20260811 14:30:53.139338 1273 raft_consensus.cc:2229] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:53.139390 648 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:53.139561 648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:53.139751 1273 raft_consensus.cc:2258] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.139947 1274 tablet_replica.cc:330] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: stopping tablet replica
I20260811 14:30:53.140038 1274 raft_consensus.cc:2229] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:53.140377 1274 raft_consensus.cc:2258] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.141031 809 tablet_service.cc:1467] Processing CreateTablet for tablet 266b1db1c66b41c99747e3d1cf288ceb (DEFAULT_TABLE table=test-table [id=471a14a4e6f642b69e675bcdab6cd787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.141023 1129 tablet_service.cc:1467] Processing CreateTablet for tablet 266b1db1c66b41c99747e3d1cf288ceb (DEFAULT_TABLE table=test-table [id=471a14a4e6f642b69e675bcdab6cd787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.141052 960 tablet_service.cc:1467] Processing CreateTablet for tablet 266b1db1c66b41c99747e3d1cf288ceb (DEFAULT_TABLE table=test-table [id=471a14a4e6f642b69e675bcdab6cd787]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.141186 960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266b1db1c66b41c99747e3d1cf288ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.141187 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266b1db1c66b41c99747e3d1cf288ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.141186 809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 266b1db1c66b41c99747e3d1cf288ceb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.142225 1248 tablet_bootstrap.cc:492] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Bootstrap starting.
I20260811 14:30:53.142603 1244 tablet_bootstrap.cc:492] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Bootstrap starting.
I20260811 14:30:53.142714 1248 tablet_bootstrap.cc:654] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.142973 1243 tablet_bootstrap.cc:492] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Bootstrap starting.
I20260811 14:30:53.143051 1244 tablet_bootstrap.cc:654] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.143443 1243 tablet_bootstrap.cc:654] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.143784 1244 tablet_bootstrap.cc:492] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: No bootstrap required, opened a new log
I20260811 14:30:53.143836 1244 ts_tablet_manager.cc:1397] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:53.144006 1244 raft_consensus.cc:348] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.144075 1244 raft_consensus.cc:374] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.144090 1244 raft_consensus.cc:729] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.144158 1244 consensus_queue.cc:260] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.144253 1244 ts_tablet_manager.cc:1428] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.144392 1274 ts_tablet_manager.cc:1905] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.146221 1273 ts_tablet_manager.cc:1905] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.146293 1274 ts_tablet_manager.cc:1918] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:53.146377 1274 log.cc:1196] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/32fdcaaeceb44d5bbb8cebf7d3965f3a
I20260811 14:30:53.146677 1274 ts_tablet_manager.cc:1939] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 4109e6aee02d4833946012e3ec4d0b91: Deleting consensus metadata
I20260811 14:30:53.147195 628 catalog_manager.cc:4841] TS 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621): tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a (table test-table [id=0fbf260ca8da4103a422556d8f9eef8f]) successfully deleted
I20260811 14:30:53.147734 1273 ts_tablet_manager.cc:1918] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:53.147830 1273 log.cc:1196] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/32fdcaaeceb44d5bbb8cebf7d3965f3a
I20260811 14:30:53.148057 1273 ts_tablet_manager.cc:1939] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 0317d40b7c6344558d4bb6aac23fa59b: Deleting consensus metadata
I20260811 14:30:53.148278 1272 tablet_replica.cc:330] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: stopping tablet replica
I20260811 14:30:53.148428 1272 raft_consensus.cc:2229] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:53.148583 1272 raft_consensus.cc:2258] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.148600 627 catalog_manager.cc:4841] TS 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499): tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a (table test-table [id=0fbf260ca8da4103a422556d8f9eef8f]) successfully deleted
I20260811 14:30:53.149224 1272 ts_tablet_manager.cc:1905] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.149228 1248 tablet_bootstrap.cc:492] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: No bootstrap required, opened a new log
I20260811 14:30:53.149302 1248 ts_tablet_manager.cc:1397] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260811 14:30:53.149468 1248 raft_consensus.cc:348] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.149564 1248 raft_consensus.cc:374] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.149587 1248 raft_consensus.cc:729] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109e6aee02d4833946012e3ec4d0b91, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.149639 1248 consensus_queue.cc:260] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.149760 1248 ts_tablet_manager.cc:1428] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:53.151160 1272 ts_tablet_manager.cc:1918] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:53.151237 1272 log.cc:1196] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/32fdcaaeceb44d5bbb8cebf7d3965f3a
I20260811 14:30:53.151257 1243 tablet_bootstrap.cc:492] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: No bootstrap required, opened a new log
I20260811 14:30:53.151298 1243 ts_tablet_manager.cc:1397] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260811 14:30:53.151506 1272 ts_tablet_manager.cc:1939] T 32fdcaaeceb44d5bbb8cebf7d3965f3a P 864cffd627ff4fc49b4387a8c03865b9: Deleting consensus metadata
I20260811 14:30:53.151472 1243 raft_consensus.cc:348] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.151544 1243 raft_consensus.cc:374] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.151571 1243 raft_consensus.cc:729] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.151621 1243 consensus_queue.cc:260] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.151728 1243 ts_tablet_manager.cc:1428] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.151986 629 catalog_manager.cc:4841] TS 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013): tablet 32fdcaaeceb44d5bbb8cebf7d3965f3a (table test-table [id=0fbf260ca8da4103a422556d8f9eef8f]) successfully deleted
W20260811 14:30:53.242600 1210 tablet.cc:2369] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:30:53.316622 1043 tablet.cc:2369] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:30:53.397497 882 tablet.cc:2369] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:30:53.429250 1249 raft_consensus.cc:482] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:53.429370 1249 raft_consensus.cc:504] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.429761 1249 leader_election.cc:288] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013)
I20260811 14:30:53.432905 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.430119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:58403 (local address 127.23.186.193:37621)
0811 14:30:53.430190 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.430193 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.430204 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:53.430237 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.430238 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.430252 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.430312 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.430499 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.431047 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.432093 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.432484 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.432540 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.432548 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.432776 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.432828 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:30:53.432919 1079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.429950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:37621 (local address 127.23.186.195:58403)
0811 14:30:53.430016 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.430032 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.430096 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.430327 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.430328 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.430330 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.430415 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.430417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.431146 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.431147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.431991 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.431994 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.432564 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.432565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.432652 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.432655 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.432761 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.432807 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:30:53.432940 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.430131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:48111 (local address 127.23.186.194:43013)
0811 14:30:53.430194 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.430196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.430273 (+ 77us) server_negotiation.cc:407] Connection header received
0811 14:30:53.430330 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.430332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.430341 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.430406 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.430659 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.431191 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.432146 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.432512 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.432608 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.432616 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.432829 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.432876 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:30:53.433018 1279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.429950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43013 (local address 127.23.186.195:48111)
0811 14:30:53.430194 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.430205 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.430273 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.430413 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.430414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.430417 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.430539 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.430541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.431290 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.431291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.432022 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.432024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.432639 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.432640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.432753 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.432758 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.432817 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.432853 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":30,"thread_start_us":110,"threads_started":1}
I20260811 14:30:53.433271 992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266b1db1c66b41c99747e3d1cf288ceb" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864cffd627ff4fc49b4387a8c03865b9" is_pre_election: true
I20260811 14:30:53.433235 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266b1db1c66b41c99747e3d1cf288ceb" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91" is_pre_election: true
I20260811 14:30:53.433391 992 raft_consensus.cc:2454] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 0.
I20260811 14:30:53.433394 829 raft_consensus.cc:2454] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 0.
I20260811 14:30:53.433590 1071 leader_election.cc:302] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0317d40b7c6344558d4bb6aac23fa59b, 4109e6aee02d4833946012e3ec4d0b91; no voters:
I20260811 14:30:53.433746 1249 raft_consensus.cc:2781] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:53.433787 1249 raft_consensus.cc:482] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:53.433813 1249 raft_consensus.cc:3037] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.434568 1249 raft_consensus.cc:504] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.434744 1249 leader_election.cc:288] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 election: Requested vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013)
I20260811 14:30:53.434875 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266b1db1c66b41c99747e3d1cf288ceb" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91"
I20260811 14:30:53.434896 992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "266b1db1c66b41c99747e3d1cf288ceb" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864cffd627ff4fc49b4387a8c03865b9"
I20260811 14:30:53.434973 829 raft_consensus.cc:3037] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.434974 992 raft_consensus.cc:3037] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.435712 992 raft_consensus.cc:2454] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 1.
I20260811 14:30:53.435742 829 raft_consensus.cc:2454] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 1.
I20260811 14:30:53.435897 1072 leader_election.cc:302] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0317d40b7c6344558d4bb6aac23fa59b, 864cffd627ff4fc49b4387a8c03865b9; no voters:
I20260811 14:30:53.436029 1249 raft_consensus.cc:2781] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:53.436175 1249 raft_consensus.cc:686] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 1 LEADER]: Becoming Leader. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Running, Role: LEADER
I20260811 14:30:53.436264 1249 consensus_queue.cc:237] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.437057 648 catalog_manager.cc:5495] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195). New cstate: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } health_report { overall_health: HEALTHY } } }
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458652, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.254@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:53.529035 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.521592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:42663 (local address 127.0.0.1:41952)
0811 14:30:53.521788 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.521799 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.521872 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.522017 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.522018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.522342 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:53.522393 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.522396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.523045 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.523047 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.523606 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.523608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.523926 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.523926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.523968 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.524586 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:53.524591 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:53.526175 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:53.527842 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:53.527850 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:53.527853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:53.528061 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:53.528248 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:53.528249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:53.528249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:53.528304 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:53.528676 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:53.528684 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:53.528763 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.528831 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.528939 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260811 14:30:53.529145 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.521715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41952 (local address 127.23.186.254:42663)
0811 14:30:53.521805 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.521807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.521864 (+ 57us) server_negotiation.cc:407] Connection header received
0811 14:30:53.521928 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.521929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.521938 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.521991 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:53.522459 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.522952 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.523683 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.523861 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.523910 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.526264 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:53.526265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:53.526265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:53.526272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:53.527751 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.528139 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.528140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.528140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.528192 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.528387 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.528389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.528389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.528593 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:53.528663 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.528865 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.529022 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":59}
I20260811 14:30:53.531525 648 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:41952:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 14:30:53.531782 648 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:53.533574 1129 tablet_service.cc:1467] Processing CreateTablet for tablet 62ff5ec03f7243a98e716c702207ae7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7c6a6c7faffb4a09bee5a1cf577351ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:53.533542 809 tablet_service.cc:1467] Processing CreateTablet for tablet 62ff5ec03f7243a98e716c702207ae7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7c6a6c7faffb4a09bee5a1cf577351ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:53.533607 960 tablet_service.cc:1467] Processing CreateTablet for tablet 62ff5ec03f7243a98e716c702207ae7f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7c6a6c7faffb4a09bee5a1cf577351ef]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 14:30:53.533735 809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ff5ec03f7243a98e716c702207ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.533733 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ff5ec03f7243a98e716c702207ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.533805 960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62ff5ec03f7243a98e716c702207ae7f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.535266 1243 tablet_bootstrap.cc:492] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: Bootstrap starting.
I20260811 14:30:53.535338 1244 tablet_bootstrap.cc:492] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: Bootstrap starting.
I20260811 14:30:53.535338 1248 tablet_bootstrap.cc:492] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: Bootstrap starting.
I20260811 14:30:53.535708 1248 tablet_bootstrap.cc:654] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.535728 1243 tablet_bootstrap.cc:654] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.535738 1244 tablet_bootstrap.cc:654] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.536430 1243 tablet_bootstrap.cc:492] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: No bootstrap required, opened a new log
I20260811 14:30:53.536446 1248 tablet_bootstrap.cc:492] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: No bootstrap required, opened a new log
I20260811 14:30:53.536497 1243 ts_tablet_manager.cc:1397] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:53.536515 1248 ts_tablet_manager.cc:1397] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:53.536517 1244 tablet_bootstrap.cc:492] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: No bootstrap required, opened a new log
I20260811 14:30:53.536571 1244 ts_tablet_manager.cc:1397] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:53.536648 1248 raft_consensus.cc:348] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.536679 1243 raft_consensus.cc:348] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.536767 1248 raft_consensus.cc:374] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.536783 1243 raft_consensus.cc:374] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.536748 1244 raft_consensus.cc:348] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.536800 1248 raft_consensus.cc:729] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109e6aee02d4833946012e3ec4d0b91, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.536808 1243 raft_consensus.cc:729] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.536818 1244 raft_consensus.cc:374] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.536844 1244 raft_consensus.cc:729] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.536844 1248 consensus_queue.cc:260] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.536863 1243 consensus_queue.cc:260] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.536896 1244 consensus_queue.cc:260] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.536953 1248 ts_tablet_manager.cc:1428] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.536980 1243 ts_tablet_manager.cc:1428] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260811 14:30:53.537002 1244 ts_tablet_manager.cc:1428] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:53.537147 1253 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.537185 1252 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.537261 1252 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.537261 1253 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.537413 1249 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.537503 1249 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.543345 1249 raft_consensus.cc:482] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:53.543454 1249 raft_consensus.cc:504] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.543612 1249 leader_election.cc:288] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013)
I20260811 14:30:53.543807 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ff5ec03f7243a98e716c702207ae7f" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91" is_pre_election: true
I20260811 14:30:53.543915 829 raft_consensus.cc:2454] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 0.
I20260811 14:30:53.543864 992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ff5ec03f7243a98e716c702207ae7f" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864cffd627ff4fc49b4387a8c03865b9" is_pre_election: true
I20260811 14:30:53.544000 992 raft_consensus.cc:2454] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 0.
I20260811 14:30:53.544108 1071 leader_election.cc:302] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0317d40b7c6344558d4bb6aac23fa59b, 4109e6aee02d4833946012e3ec4d0b91; no voters:
I20260811 14:30:53.544220 1249 raft_consensus.cc:2781] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:53.544260 1249 raft_consensus.cc:482] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:53.544291 1249 raft_consensus.cc:3037] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.545122 1249 raft_consensus.cc:504] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.545272 1249 leader_election.cc:288] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 election: Requested vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013)
I20260811 14:30:53.545426 992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ff5ec03f7243a98e716c702207ae7f" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "864cffd627ff4fc49b4387a8c03865b9"
I20260811 14:30:53.545435 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62ff5ec03f7243a98e716c702207ae7f" candidate_uuid: "0317d40b7c6344558d4bb6aac23fa59b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91"
I20260811 14:30:53.545534 992 raft_consensus.cc:3037] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.545532 829 raft_consensus.cc:3037] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.546150 829 raft_consensus.cc:2454] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 1.
I20260811 14:30:53.546218 992 raft_consensus.cc:2454] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0317d40b7c6344558d4bb6aac23fa59b in term 1.
I20260811 14:30:53.546324 1071 leader_election.cc:302] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0317d40b7c6344558d4bb6aac23fa59b, 4109e6aee02d4833946012e3ec4d0b91; no voters:
I20260811 14:30:53.546417 1249 raft_consensus.cc:2781] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:53.546483 1249 raft_consensus.cc:686] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [term 1 LEADER]: Becoming Leader. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Running, Role: LEADER
I20260811 14:30:53.546535 1249 consensus_queue.cc:237] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.547003 1249 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: TxnStatusTablet state changed. Reason: New leader 0317d40b7c6344558d4bb6aac23fa59b. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.547102 1249 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:53.547264 1296 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 14:30:53.547420 1296 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 14:30:53.547649 648 catalog_manager.cc:5495] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195). New cstate: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } health_report { overall_health: HEALTHY } } }
I20260811 14:30:53.553611 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.550612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55700 (local address 127.23.186.195:36499)
0811 14:30:53.550653 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.550655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.550714 (+ 59us) server_negotiation.cc:407] Connection header received
0811 14:30:53.550873 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.550874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.550889 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.550947 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:53.551076 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.551561 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.552809 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.553005 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.553064 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.553280 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.553379 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.553428 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.553516 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.553534 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:30:53.553668 1289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.550542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36499 (local address 127.0.0.1:55700)
0811 14:30:53.550648 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.550662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.550728 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.550953 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.550954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.550956 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:53.551013 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.551015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.551650 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.551651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.552595 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.552598 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.553087 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.553088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.553155 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.553176 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.553443 (+ 267us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.553453 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.553514 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.553575 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 14:30:53.555007 992 raft_consensus.cc:1261] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Refusing update from remote peer 0317d40b7c6344558d4bb6aac23fa59b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.555097 829 raft_consensus.cc:1261] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Refusing update from remote peer 0317d40b7c6344558d4bb6aac23fa59b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.555351 1249 consensus_queue.cc:1035] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [LEADER]: Connected to new peer: Peer: permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.555505 1280 consensus_queue.cc:1035] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.556178 1252 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: TxnStatusTablet state changed. Reason: New leader 0317d40b7c6344558d4bb6aac23fa59b. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.556281 1252 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.557292 1249 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.557382 1249 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:53.557379 1253 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: TxnStatusTablet state changed. Reason: New leader 0317d40b7c6344558d4bb6aac23fa59b. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.557471 1253 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.557544 1252 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.557631 1252 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 864cffd627ff4fc49b4387a8c03865b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.558432 1280 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.558539 1280 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 0317d40b7c6344558d4bb6aac23fa59b: This TxnStatusTablet replica's current role is: LEADER
I20260811 14:30:53.558753 1253 tablet_replica.cc:439] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0317d40b7c6344558d4bb6aac23fa59b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } } }
I20260811 14:30:53.558851 1253 tablet_replica.cc:442] T 62ff5ec03f7243a98e716c702207ae7f P 4109e6aee02d4833946012e3ec4d0b91: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 14:30:53.562003 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.559662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.23.186.195:36499)
0811 14:30:53.559700 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.559702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.559735 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:30:53.559811 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.559812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.559823 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.559879 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:53.559981 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.560495 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.561307 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.561515 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.561597 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.561741 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.561776 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.561839 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.561907 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.561927 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:30:53.567679 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.565005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:32951 (local address 127.23.186.195:36499)
0811 14:30:53.565049 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.565051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.565179 (+ 128us) server_negotiation.cc:407] Connection header received
0811 14:30:53.565207 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.565208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.565217 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.565279 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:53.565415 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.566139 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.566858 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.567157 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.567231 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.567417 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.567453 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.567506 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.567557 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.567576 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:53.567701 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.564920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36499 (local address 127.23.186.195:32951)
0811 14:30:53.565103 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.565115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.565172 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.565288 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.565289 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.565290 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:53.565356 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.565357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.566219 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.566219 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.566781 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.566782 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.567254 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.567255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.567312 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.567328 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.567516 (+ 188us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.567518 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.567554 (+ 36us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.567592 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":135,"threads_started":1}
I20260811 14:30:53.570361 992 raft_consensus.cc:1261] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Refusing update from remote peer 0317d40b7c6344558d4bb6aac23fa59b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.570389 829 raft_consensus.cc:1261] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Refusing update from remote peer 0317d40b7c6344558d4bb6aac23fa59b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.570642 1280 consensus_queue.cc:1035] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [LEADER]: Connected to new peer: Peer: permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.570765 1302 consensus_queue.cc:1035] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.589565 646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:53.589694 646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:53.590361 646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Sending DeleteTablet for 3 replicas of tablet 266b1db1c66b41c99747e3d1cf288ceb
I20260811 14:30:53.590653 1129 tablet_service.cc:1514] Processing DeleteTablet for tablet 266b1db1c66b41c99747e3d1cf288ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:55690
I20260811 14:30:53.590683 809 tablet_service.cc:1514] Processing DeleteTablet for tablet 266b1db1c66b41c99747e3d1cf288ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43104
I20260811 14:30:53.590766 1273 tablet_replica.cc:330] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: stopping tablet replica
I20260811 14:30:53.590827 1274 tablet_replica.cc:330] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: stopping tablet replica
I20260811 14:30:53.590837 1273 raft_consensus.cc:2229] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:53.590893 1274 raft_consensus.cc:2229] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:53.590945 1274 raft_consensus.cc:2258] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.590912 960 tablet_service.cc:1514] Processing DeleteTablet for tablet 266b1db1c66b41c99747e3d1cf288ceb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:34594
I20260811 14:30:53.590945 1273 raft_consensus.cc:2258] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.591120 1272 tablet_replica.cc:330] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: stopping tablet replica
I20260811 14:30:53.591193 1272 raft_consensus.cc:2229] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:53.591236 1272 raft_consensus.cc:2258] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.591318 1273 ts_tablet_manager.cc:1905] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.591691 1272 ts_tablet_manager.cc:1905] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.593619 1272 ts_tablet_manager.cc:1918] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:53.593681 1272 log.cc:1196] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/266b1db1c66b41c99747e3d1cf288ceb
I20260811 14:30:53.593926 1272 ts_tablet_manager.cc:1939] T 266b1db1c66b41c99747e3d1cf288ceb P 864cffd627ff4fc49b4387a8c03865b9: Deleting consensus metadata
I20260811 14:30:53.594755 629 catalog_manager.cc:4841] TS 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013): tablet 266b1db1c66b41c99747e3d1cf288ceb (table test-table [id=471a14a4e6f642b69e675bcdab6cd787]) successfully deleted
I20260811 14:30:53.594848 1273 ts_tablet_manager.cc:1918] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:53.594907 1273 log.cc:1196] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/266b1db1c66b41c99747e3d1cf288ceb
I20260811 14:30:53.595131 1273 ts_tablet_manager.cc:1939] T 266b1db1c66b41c99747e3d1cf288ceb P 0317d40b7c6344558d4bb6aac23fa59b: Deleting consensus metadata
I20260811 14:30:53.595494 627 catalog_manager.cc:4841] TS 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499): tablet 266b1db1c66b41c99747e3d1cf288ceb (table test-table [id=471a14a4e6f642b69e675bcdab6cd787]) successfully deleted
I20260811 14:30:53.598085 1274 ts_tablet_manager.cc:1905] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.600020 1274 ts_tablet_manager.cc:1918] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:53.600104 1274 log.cc:1196] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/266b1db1c66b41c99747e3d1cf288ceb
I20260811 14:30:53.600351 1274 ts_tablet_manager.cc:1939] T 266b1db1c66b41c99747e3d1cf288ceb P 4109e6aee02d4833946012e3ec4d0b91: Deleting consensus metadata
I20260811 14:30:53.600766 628 catalog_manager.cc:4841] TS 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621): tablet 266b1db1c66b41c99747e3d1cf288ceb (table test-table [id=471a14a4e6f642b69e675bcdab6cd787]) successfully deleted
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:30:53.617753 716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:53.611850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.23.186.254:42663)
0811 14:30:53.611925 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.611928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.612036 (+ 108us) server_negotiation.cc:407] Connection header received
0811 14:30:53.612152 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.612153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.612172 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.612239 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:53.612909 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.613417 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.614183 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.614379 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.614470 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.617667 (+ 3197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41968: BlockingRecv error: recv got EOF from 127.0.0.1:41968 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 11 14:30:53 dist-test-slave-cbsf krb5kdc[594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM
I20260811 14:30:53.626052 716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.618734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41978 (local address 127.23.186.254:42663)
0811 14:30:53.618778 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.618780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.618787 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:53.618850 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.618851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.618861 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.618915 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:53.619463 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.619984 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.620815 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.621003 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.621082 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.623303 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:53.623305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:53.623305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:53.623320 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:53.624844 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.625200 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.625201 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.625202 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.625262 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:53.625466 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:53.625467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:53.625467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:53.625602 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:53.625705 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.625850 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.625935 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:30:53.627696 646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:53.627889 646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:53.629464 960 tablet_service.cc:1467] Processing CreateTablet for tablet dfb7837a3a584893a12390880d817b65 (DEFAULT_TABLE table=test-table [id=f58ef8507a324972baa35c0fe13f8beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.629490 1129 tablet_service.cc:1467] Processing CreateTablet for tablet dfb7837a3a584893a12390880d817b65 (DEFAULT_TABLE table=test-table [id=f58ef8507a324972baa35c0fe13f8beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.629535 809 tablet_service.cc:1467] Processing CreateTablet for tablet dfb7837a3a584893a12390880d817b65 (DEFAULT_TABLE table=test-table [id=f58ef8507a324972baa35c0fe13f8beb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.629631 960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb7837a3a584893a12390880d817b65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.629640 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb7837a3a584893a12390880d817b65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.629664 809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfb7837a3a584893a12390880d817b65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.631002 1248 tablet_bootstrap.cc:492] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Bootstrap starting.
I20260811 14:30:53.631002 1243 tablet_bootstrap.cc:492] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Bootstrap starting.
I20260811 14:30:53.631129 1244 tablet_bootstrap.cc:492] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Bootstrap starting.
I20260811 14:30:53.631381 1248 tablet_bootstrap.cc:654] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.631418 1243 tablet_bootstrap.cc:654] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.631422 1244 tablet_bootstrap.cc:654] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.632129 1248 tablet_bootstrap.cc:492] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: No bootstrap required, opened a new log
I20260811 14:30:53.632184 1248 ts_tablet_manager.cc:1397] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:53.632202 1243 tablet_bootstrap.cc:492] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: No bootstrap required, opened a new log
I20260811 14:30:53.632251 1243 ts_tablet_manager.cc:1397] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:53.632320 1248 raft_consensus.cc:348] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.632401 1248 raft_consensus.cc:374] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.632424 1248 raft_consensus.cc:729] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109e6aee02d4833946012e3ec4d0b91, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.632412 1243 raft_consensus.cc:348] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.632467 1248 consensus_queue.cc:260] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.632490 1243 raft_consensus.cc:374] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.632510 1243 raft_consensus.cc:729] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.632560 1248 ts_tablet_manager.cc:1428] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:53.632570 1243 consensus_queue.cc:260] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.632658 1243 ts_tablet_manager.cc:1428] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.637252 1244 tablet_bootstrap.cc:492] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: No bootstrap required, opened a new log
I20260811 14:30:53.637323 1244 ts_tablet_manager.cc:1397] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260811 14:30:53.637481 1244 raft_consensus.cc:348] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.637569 1244 raft_consensus.cc:374] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.637591 1244 raft_consensus.cc:729] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.637674 1244 consensus_queue.cc:260] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.637781 1244 ts_tablet_manager.cc:1428] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.743331 1252 raft_consensus.cc:482] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:53.743469 1252 raft_consensus.cc:504] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.743894 1252 leader_election.cc:288] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:53.747771 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.744345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:45595 (local address 127.23.186.193:37621)
0811 14:30:53.744438 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.744440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.744449 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:53.744481 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.744482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.744491 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.744553 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.744770 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.745250 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.746787 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.747376 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.747455 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.747465 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.747650 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.747696 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 14:30:53.747784 1329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.744089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:37621 (local address 127.23.186.194:45595)
0811 14:30:53.744236 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.744250 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.744321 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.744610 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.744610 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.744613 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.744669 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.744671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.745346 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.745347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.746672 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.746676 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.747485 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.747486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.747556 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.747561 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.747635 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.747684 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":84,"threads_started":1}
I20260811 14:30:53.748087 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.744241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:35469 (local address 127.23.186.195:36499)
0811 14:30:53.744292 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.744294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.744334 (+ 40us) server_negotiation.cc:407] Connection header received
0811 14:30:53.744441 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.744442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.744453 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.744483 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:30:53.744734 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.745484 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.747057 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.747587 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.747650 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.747658 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.747970 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.748012 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:30:53.748065 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb7837a3a584893a12390880d817b65" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91" is_pre_election: true
I20260811 14:30:53.748172 829 raft_consensus.cc:2454] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 0.
I20260811 14:30:53.748338 1331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.744137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36499 (local address 127.23.186.194:35469)
0811 14:30:53.744278 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:53.744287 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:53.744359 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:53.744553 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:53.744554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:53.744556 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:30:53.744640 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.744643 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.745578 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.745579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.746926 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:53.746928 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.747689 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.747690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:53.747752 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.747755 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:53.748205 (+ 450us) client_negotiation.cc:240] Negotiation successful
0811 14:30:53.748254 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":1,"thread_start_us":56,"threads_started":1}
I20260811 14:30:53.748399 910 leader_election.cc:302] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109e6aee02d4833946012e3ec4d0b91, 864cffd627ff4fc49b4387a8c03865b9; no voters:
I20260811 14:30:53.748534 1252 raft_consensus.cc:2781] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:53.748569 1252 raft_consensus.cc:482] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:53.748587 1252 raft_consensus.cc:3037] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.748605 1150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb7837a3a584893a12390880d817b65" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317d40b7c6344558d4bb6aac23fa59b" is_pre_election: true
I20260811 14:30:53.748713 1150 raft_consensus.cc:2454] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 0.
I20260811 14:30:53.749425 1252 raft_consensus.cc:504] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.749620 1252 leader_election.cc:288] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 election: Requested vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:53.749791 1150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb7837a3a584893a12390880d817b65" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317d40b7c6344558d4bb6aac23fa59b"
I20260811 14:30:53.749792 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfb7837a3a584893a12390880d817b65" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91"
I20260811 14:30:53.749884 1150 raft_consensus.cc:3037] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.749903 829 raft_consensus.cc:3037] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:53.750725 829 raft_consensus.cc:2454] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 1.
I20260811 14:30:53.750845 1150 raft_consensus.cc:2454] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 1.
I20260811 14:30:53.750914 910 leader_election.cc:302] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109e6aee02d4833946012e3ec4d0b91, 864cffd627ff4fc49b4387a8c03865b9; no voters:
I20260811 14:30:53.751022 1252 raft_consensus.cc:2781] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:53.751206 1252 raft_consensus.cc:686] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 1 LEADER]: Becoming Leader. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Running, Role: LEADER
I20260811 14:30:53.751293 1252 consensus_queue.cc:237] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.752046 645 catalog_manager.cc:5495] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "864cffd627ff4fc49b4387a8c03865b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:53.781425 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:53.778503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34600 (local address 127.23.186.194:43013)
0811 14:30:53.778548 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:53.778550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:53.778562 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:53.778665 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:53.778666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:53.778693 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:53.778767 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:53.778952 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.779442 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.780545 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:53.780810 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:53.780879 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:53.781054 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:53.781168 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:53.781225 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:53.781294 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 14:30:53.781317 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:30:53.782667 1150 raft_consensus.cc:1261] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Refusing update from remote peer 864cffd627ff4fc49b4387a8c03865b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.782671 829 raft_consensus.cc:1261] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Refusing update from remote peer 864cffd627ff4fc49b4387a8c03865b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:53.782966 1252 consensus_queue.cc:1035] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.783110 1332 consensus_queue.cc:1035] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:53.787285 646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:53.787400 646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:53.788121 646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Sending DeleteTablet for 3 replicas of tablet dfb7837a3a584893a12390880d817b65
I20260811 14:30:53.788405 809 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb7837a3a584893a12390880d817b65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43104
I20260811 14:30:53.788532 1274 tablet_replica.cc:330] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: stopping tablet replica
I20260811 14:30:53.788575 960 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb7837a3a584893a12390880d817b65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:34594
I20260811 14:30:53.788600 1274 raft_consensus.cc:2229] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:53.788658 1274 raft_consensus.cc:2258] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.788772 1272 tablet_replica.cc:330] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: stopping tablet replica
I20260811 14:30:53.788832 1272 raft_consensus.cc:2229] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:53.788918 1272 raft_consensus.cc:2258] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.788676 1129 tablet_service.cc:1514] Processing DeleteTablet for tablet dfb7837a3a584893a12390880d817b65 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:53 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:55690
I20260811 14:30:53.789183 1273 tablet_replica.cc:330] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: stopping tablet replica
I20260811 14:30:53.789242 1273 raft_consensus.cc:2229] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:53.789286 1273 raft_consensus.cc:2258] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:53.789721 645 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41978:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:30:53.789884 645 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:30:53.790210 1272 ts_tablet_manager.cc:1905] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.791301 809 tablet_service.cc:1467] Processing CreateTablet for tablet d508337ed56a499eae1772f824b7d97d (DEFAULT_TABLE table=test-table [id=e16814d7230d4c558f89d9fbc66cb5e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.791339 1129 tablet_service.cc:1467] Processing CreateTablet for tablet d508337ed56a499eae1772f824b7d97d (DEFAULT_TABLE table=test-table [id=e16814d7230d4c558f89d9fbc66cb5e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.791458 809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d508337ed56a499eae1772f824b7d97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.791460 1129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d508337ed56a499eae1772f824b7d97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.791826 1272 ts_tablet_manager.cc:1918] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:53.791882 1272 log.cc:1196] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/dfb7837a3a584893a12390880d817b65
I20260811 14:30:53.792116 1272 ts_tablet_manager.cc:1939] T dfb7837a3a584893a12390880d817b65 P 864cffd627ff4fc49b4387a8c03865b9: Deleting consensus metadata
I20260811 14:30:53.792840 1273 ts_tablet_manager.cc:1905] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.792999 1248 tablet_bootstrap.cc:492] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91: Bootstrap starting.
I20260811 14:30:53.792999 1244 tablet_bootstrap.cc:492] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Bootstrap starting.
I20260811 14:30:53.793386 1248 tablet_bootstrap.cc:654] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.793447 1244 tablet_bootstrap.cc:654] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.794291 1273 ts_tablet_manager.cc:1918] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:53.794360 1273 log.cc:1196] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/dfb7837a3a584893a12390880d817b65
I20260811 14:30:53.794577 1273 ts_tablet_manager.cc:1939] T dfb7837a3a584893a12390880d817b65 P 0317d40b7c6344558d4bb6aac23fa59b: Deleting consensus metadata
I20260811 14:30:53.794631 1244 tablet_bootstrap.cc:492] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: No bootstrap required, opened a new log
I20260811 14:30:53.794697 1244 ts_tablet_manager.cc:1397] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:30:53.794826 1244 raft_consensus.cc:348] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.794868 960 tablet_service.cc:1467] Processing CreateTablet for tablet d508337ed56a499eae1772f824b7d97d (DEFAULT_TABLE table=test-table [id=e16814d7230d4c558f89d9fbc66cb5e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:30:53.794902 1244 raft_consensus.cc:374] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.794929 1244 raft_consensus.cc:729] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0317d40b7c6344558d4bb6aac23fa59b, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.794974 960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d508337ed56a499eae1772f824b7d97d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:53.794978 1244 consensus_queue.cc:260] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.795079 1244 ts_tablet_manager.cc:1428] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.796212 1243 tablet_bootstrap.cc:492] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Bootstrap starting.
I20260811 14:30:53.796331 627 catalog_manager.cc:4841] TS 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499): tablet dfb7837a3a584893a12390880d817b65 (table test-table [id=f58ef8507a324972baa35c0fe13f8beb]) successfully deleted
I20260811 14:30:53.796681 1243 tablet_bootstrap.cc:654] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:53.797371 629 catalog_manager.cc:4841] TS 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013): tablet dfb7837a3a584893a12390880d817b65 (table test-table [id=f58ef8507a324972baa35c0fe13f8beb]) successfully deleted
I20260811 14:30:53.797674 1243 tablet_bootstrap.cc:492] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: No bootstrap required, opened a new log
I20260811 14:30:53.797745 1243 ts_tablet_manager.cc:1397] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:30:53.797821 1274 ts_tablet_manager.cc:1905] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:53.797910 1243 raft_consensus.cc:348] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.798002 1243 raft_consensus.cc:374] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.798026 1243 raft_consensus.cc:729] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.798074 1243 consensus_queue.cc:260] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.798173 1243 ts_tablet_manager.cc:1428] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:53.799264 1274 ts_tablet_manager.cc:1918] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:30:53.799329 1274 log.cc:1196] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/dfb7837a3a584893a12390880d817b65
I20260811 14:30:53.799535 1274 ts_tablet_manager.cc:1939] T dfb7837a3a584893a12390880d817b65 P 4109e6aee02d4833946012e3ec4d0b91: Deleting consensus metadata
I20260811 14:30:53.799993 628 catalog_manager.cc:4841] TS 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621): tablet dfb7837a3a584893a12390880d817b65 (table test-table [id=f58ef8507a324972baa35c0fe13f8beb]) successfully deleted
I20260811 14:30:53.808151 1248 tablet_bootstrap.cc:492] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91: No bootstrap required, opened a new log
I20260811 14:30:53.808243 1248 ts_tablet_manager.cc:1397] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91: Time spent bootstrapping tablet: real 0.015s user 0.001s sys 0.000s
I20260811 14:30:53.808399 1248 raft_consensus.cc:348] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.808499 1248 raft_consensus.cc:374] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:53.808525 1248 raft_consensus.cc:729] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4109e6aee02d4833946012e3ec4d0b91, State: Initialized, Role: FOLLOWER
I20260811 14:30:53.808583 1248 consensus_queue.cc:260] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:53.808735 1248 ts_tablet_manager.cc:1428] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260811 14:30:54.004165 1337 raft_consensus.cc:482] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:30:54.004313 1337 raft_consensus.cc:504] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:54.004534 1337 leader_election.cc:288] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:54.004850 1150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d508337ed56a499eae1772f824b7d97d" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317d40b7c6344558d4bb6aac23fa59b" is_pre_election: true
I20260811 14:30:54.004945 1150 raft_consensus.cc:2454] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 0.
I20260811 14:30:54.004863 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d508337ed56a499eae1772f824b7d97d" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91" is_pre_election: true
I20260811 14:30:54.005008 829 raft_consensus.cc:2454] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 0.
I20260811 14:30:54.005132 909 leader_election.cc:302] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0317d40b7c6344558d4bb6aac23fa59b, 864cffd627ff4fc49b4387a8c03865b9; no voters:
I20260811 14:30:54.005255 1337 raft_consensus.cc:2781] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:30:54.005327 1337 raft_consensus.cc:482] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:30:54.005350 1337 raft_consensus.cc:3037] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:54.006147 1337 raft_consensus.cc:504] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:54.006302 1337 leader_election.cc:288] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 election: Requested vote from peers 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621), 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499)
I20260811 14:30:54.006485 829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d508337ed56a499eae1772f824b7d97d" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4109e6aee02d4833946012e3ec4d0b91"
I20260811 14:30:54.006563 829 raft_consensus.cc:3037] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:54.006543 1150 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d508337ed56a499eae1772f824b7d97d" candidate_uuid: "864cffd627ff4fc49b4387a8c03865b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0317d40b7c6344558d4bb6aac23fa59b"
I20260811 14:30:54.006649 1150 raft_consensus.cc:3037] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:54.007313 829 raft_consensus.cc:2454] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 1.
I20260811 14:30:54.007411 1150 raft_consensus.cc:2454] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 864cffd627ff4fc49b4387a8c03865b9 in term 1.
I20260811 14:30:54.007499 910 leader_election.cc:302] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4109e6aee02d4833946012e3ec4d0b91, 864cffd627ff4fc49b4387a8c03865b9; no voters:
I20260811 14:30:54.007594 1337 raft_consensus.cc:2781] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:54.007640 1337 raft_consensus.cc:686] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 1 LEADER]: Becoming Leader. State: Replica: 864cffd627ff4fc49b4387a8c03865b9, State: Running, Role: LEADER
I20260811 14:30:54.007719 1337 consensus_queue.cc:237] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } }
I20260811 14:30:54.008424 645 catalog_manager.cc:5495] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "864cffd627ff4fc49b4387a8c03865b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "864cffd627ff4fc49b4387a8c03865b9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43013 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 } health_report { overall_health: UNKNOWN } } }
I20260811 14:30:54.031163 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:54.028679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:52729 (local address 127.23.186.195:36499)
0811 14:30:54.028781 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:54.028783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:54.028871 (+ 88us) server_negotiation.cc:407] Connection header received
0811 14:30:54.028943 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:54.028944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:54.028955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:54.029011 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:54.029146 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.029652 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.030465 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.030664 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.030737 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:54.030913 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:54.030951 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:54.031013 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:54.031066 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 14:30:54.031088 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 14:30:54.031195 1350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:54.028617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36499 (local address 127.23.186.194:52729)
0811 14:30:54.028789 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:54.028801 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:54.028890 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:54.029015 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:54.029016 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:54.029017 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:54.029072 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:54.029074 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.029767 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.029768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:54.030368 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:54.030369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.030736 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.030737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:54.030813 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:54.030828 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:54.031014 (+ 186us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:54.031017 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:30:54.031064 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 14:30:54.031115 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":94,"threads_started":1}
I20260811 14:30:54.033260 1150 raft_consensus.cc:1261] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Refusing update from remote peer 864cffd627ff4fc49b4387a8c03865b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:54.033260 829 raft_consensus.cc:1261] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Refusing update from remote peer 864cffd627ff4fc49b4387a8c03865b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:30:54.033514 1252 consensus_queue.cc:1035] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0317d40b7c6344558d4bb6aac23fa59b" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:54.033610 1337 consensus_queue.cc:1035] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4109e6aee02d4833946012e3ec4d0b91" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 37621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:30:54.041185 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:54.038977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:51913 (local address 127.23.186.194:43013)
0811 14:30:54.039020 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:54.039022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:54.039031 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:54.039071 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:54.039072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:54.039081 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:54.039119 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:30:54.039202 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.039662 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.040419 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.040662 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.040795 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:54.040943 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:30:54.040983 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:30:54.041046 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:54.041111 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:30:54.041131 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:30:54.041211 1305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:54.038914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43013 (local address 127.23.186.195:51913)
0811 14:30:54.038957 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:54.038971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:54.039010 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:54.039121 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:54.039122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:54.039123 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 14:30:54.039162 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:54.039163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.039721 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.039721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:54.040259 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:54.040260 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:54.040794 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:54.040795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:54.040860 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:54.040867 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 14:30:54.041062 (+ 195us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 14:30:54.041064 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:30:54.041109 (+ 45us) client_negotiation.cc:240] Negotiation successful
0811 14:30:54.041156 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 14:30:54.049470 645 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:54.049602 645 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41978:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:30:54.050266 645 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4e44cf3244a48a99d40404370f7c604: Sending DeleteTablet for 3 replicas of tablet d508337ed56a499eae1772f824b7d97d
I20260811 14:30:54.050578 1129 tablet_service.cc:1514] Processing DeleteTablet for tablet d508337ed56a499eae1772f824b7d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:54 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:55690
I20260811 14:30:54.050578 960 tablet_service.cc:1514] Processing DeleteTablet for tablet d508337ed56a499eae1772f824b7d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:54 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:34594
I20260811 14:30:54.050587 809 tablet_service.cc:1514] Processing DeleteTablet for tablet d508337ed56a499eae1772f824b7d97d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:30:54 UTC) from {username='oryx', principal='oryx/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43104
I20260811 14:30:54.050684 1272 tablet_replica.cc:330] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: stopping tablet replica
I20260811 14:30:54.050684 1273 tablet_replica.cc:330] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: stopping tablet replica
I20260811 14:30:54.050689 1274 tablet_replica.cc:330] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91: stopping tablet replica
I20260811 14:30:54.050729 1273 raft_consensus.cc:2229] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:54.050729 1272 raft_consensus.cc:2229] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:30:54.050745 1274 raft_consensus.cc:2229] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:30:54.050765 1273 raft_consensus.cc:2258] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:54.050786 1274 raft_consensus.cc:2258] T d508337ed56a499eae1772f824b7d97d P 4109e6aee02d4833946012e3ec4d0b91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:54.050798 1272 raft_consensus.cc:2258] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:30:54.050978 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 730
I20260811 14:30:54.051276 1272 ts_tablet_manager.cc:1905] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:54.051291 1273 ts_tablet_manager.cc:1905] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:30:54.052852 1273 ts_tablet_manager.cc:1918] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:54.052917 1273 log.cc:1196] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/d508337ed56a499eae1772f824b7d97d
I20260811 14:30:54.053136 1273 ts_tablet_manager.cc:1939] T d508337ed56a499eae1772f824b7d97d P 0317d40b7c6344558d4bb6aac23fa59b: Deleting consensus metadata
I20260811 14:30:54.053352 1272 ts_tablet_manager.cc:1918] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 14:30:54.053426 1272 log.cc:1196] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/d508337ed56a499eae1772f824b7d97d
I20260811 14:30:54.053498 627 catalog_manager.cc:4841] TS 0317d40b7c6344558d4bb6aac23fa59b (127.23.186.195:36499): tablet d508337ed56a499eae1772f824b7d97d (table test-table [id=e16814d7230d4c558f89d9fbc66cb5e6]) successfully deleted
I20260811 14:30:54.053637 1272 ts_tablet_manager.cc:1939] T d508337ed56a499eae1772f824b7d97d P 864cffd627ff4fc49b4387a8c03865b9: Deleting consensus metadata
I20260811 14:30:54.054374 629 catalog_manager.cc:4841] TS 864cffd627ff4fc49b4387a8c03865b9 (127.23.186.194:43013): tablet d508337ed56a499eae1772f824b7d97d (table test-table [id=e16814d7230d4c558f89d9fbc66cb5e6]) successfully deleted
W20260811 14:30:54.062994 628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:37621 (error 108)
I20260811 14:30:54.063364 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 891
I20260811 14:30:54.065477 1356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:54.063481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:37621 (local address 127.0.0.1:43118)
0811 14:30:54.065342 (+ 1861us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:54.065423 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:37621: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1837,"thread_start_us":43,"threads_started":1}
W20260811 14:30:54.065590 628 catalog_manager.cc:4584] TS 4109e6aee02d4833946012e3ec4d0b91 (127.23.186.193:37621): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d508337ed56a499eae1772f824b7d97d: Network error: Client connection negotiation failed: client connection to 127.23.186.193:37621: connect: Connection refused (error 111)
I20260811 14:30:54.073724 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 1051
I20260811 14:30:54.082809 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 608
2026-08-11T14:30:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3024 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:54 dist-test-slave-cbsf krb5kdc[1361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:54 dist-test-slave-cbsf krb5kdc[1361](info): set up 2 sockets
Aug 11 14:30:54 dist-test-slave-cbsf krb5kdc[1361](info): commencing operation
krb5kdc: starting...
W20260811 14:30:55.237519 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:55 dist-test-slave-cbsf krb5kdc[1361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:55.371246 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44483
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:39539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:44483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:55.455853 1377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:55.456122 1377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:55.456161 1377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:55.457372 1377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:55.457456 1377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:55.457484 1377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:55.457516 1377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:55.457537 1377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:55.459054 1377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:39539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:44483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:55.459352 1377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:55.459599 1377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:55.464608 1383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:55.464635 1384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:55.464655 1386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:55.464947 1377 server_base.cc:1034] running on GCE node
I20260811 14:30:55.465171 1377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:55.465420 1377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:55.466607 1377 hybrid_clock.cc:648] HybridClock initialized: now 1786458655466602 us; error 38 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T14:30:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1383 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:30:55 dist-test-slave-cbsf krb5kdc[1392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:30:55 dist-test-slave-cbsf krb5kdc[1392](info): set up 2 sockets
Aug 11 14:30:55 dist-test-slave-cbsf krb5kdc[1392](info): commencing operation
krb5kdc: starting...
W20260811 14:30:56.591814 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:30:56 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:30:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:30:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:30:56.725476 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33769
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:33769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:56.810115 1409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:56.810386 1409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:56.810424 1409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:56.811580 1409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:56.811648 1409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:56.811672 1409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:56.811699 1409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:56.811734 1409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:56.813282 1409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:33769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:56.813585 1409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:56.813827 1409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:56.819154 1419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:56.819133 1416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:56.819317 1409 server_base.cc:1034] running on GCE node
W20260811 14:30:56.819199 1417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:56.819653 1409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:56.819967 1409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:56.821116 1409 hybrid_clock.cc:648] HybridClock initialized: now 1786458656821102 us; error 31 us; skew 500 ppm
Aug 11 14:30:56 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458656, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:56.829424 1409 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:56.829923 1409 webserver.cc:460] Webserver started at http://127.23.186.254:45481/ using document root <none> and password file <none>
I20260811 14:30:56.830195 1409 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:56.830271 1409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:56.830431 1409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:56.831795 1409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "e257179d07234303b5116438d114182d"
format_stamp: "Formatted at 2026-08-11 14:30:56 on dist-test-slave-cbsf"
server_key: "34757b78e7a019db4548efb7f00914b2"
server_key_iv: "01b080220ced95cd4599f09289e1f2bf"
server_key_version: "encryptionkey@0"
I20260811 14:30:56.832208 1409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "e257179d07234303b5116438d114182d"
format_stamp: "Formatted at 2026-08-11 14:30:56 on dist-test-slave-cbsf"
server_key: "34757b78e7a019db4548efb7f00914b2"
server_key_iv: "01b080220ced95cd4599f09289e1f2bf"
server_key_version: "encryptionkey@0"
I20260811 14:30:56.833798 1409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 14:30:56.834681 1429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:56.834903 1409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:56.835017 1409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "e257179d07234303b5116438d114182d"
format_stamp: "Formatted at 2026-08-11 14:30:56 on dist-test-slave-cbsf"
server_key: "34757b78e7a019db4548efb7f00914b2"
server_key_iv: "01b080220ced95cd4599f09289e1f2bf"
server_key_version: "encryptionkey@0"
I20260811 14:30:56.835100 1409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:56.880968 1409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:56.883577 1409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:56.883765 1409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:56.887971 1409 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:33769
I20260811 14:30:56.888016 1491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:33769 every 8 connection(s)
I20260811 14:30:56.888439 1409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:56.889439 1493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:30:56.891968 1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Bootstrap starting.
I20260811 14:30:56.892043 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 1409
I20260811 14:30:56.892186 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:30:56.892366 24299 external_mini_cluster.cc:1428] Setting key 1e5f5152cd8a33f16f62c59dda233e98
I20260811 14:30:56.892803 1493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Neither blocks nor log segments found. Creating new log.
I20260811 14:30:56.893276 1493 log.cc:824] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:56.894093 1493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: No bootstrap required, opened a new log
I20260811 14:30:56.896546 1493 raft_consensus.cc:348] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:56.896684 1493 raft_consensus.cc:374] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:30:56.896781 1493 raft_consensus.cc:729] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e257179d07234303b5116438d114182d, State: Initialized, Role: FOLLOWER
I20260811 14:30:56.896925 1493 consensus_queue.cc:260] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:56.897012 1493 raft_consensus.cc:388] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:56.897060 1493 raft_consensus.cc:482] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:56.897110 1493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:30:56.897720 1493 raft_consensus.cc:504] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:56.897848 1493 leader_election.cc:302] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e257179d07234303b5116438d114182d; no voters:
I20260811 14:30:56.898052 1493 leader_election.cc:288] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:30:56.898116 1499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:30:56.898334 1493 sys_catalog.cc:564] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:56.899379 1499 raft_consensus.cc:686] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 LEADER]: Becoming Leader. State: Replica: e257179d07234303b5116438d114182d, State: Running, Role: LEADER
I20260811 14:30:56.899565 1499 consensus_queue.cc:237] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
Aug 11 14:30:56 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458656, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:56.900138 1500 sys_catalog.cc:455] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e257179d07234303b5116438d114182d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } } }
I20260811 14:30:56.900223 1500 sys_catalog.cc:458] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:56.900615 1501 sys_catalog.cc:455] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e257179d07234303b5116438d114182d. Latest consensus state: current_term: 1 leader_uuid: "e257179d07234303b5116438d114182d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } } }
I20260811 14:30:56.900717 1501 sys_catalog.cc:458] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: This master's current role is: LEADER
W20260811 14:30:56.901264 1518 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:30:56.901360 1518 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:30:56.901463 1509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:56.901649 1509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:56.903342 1509 catalog_manager.cc:1347] Generated new cluster ID: f2cebd6db2644b4785db85442b86d99a
I20260811 14:30:56.903395 1509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:56.904983 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:56.892949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60600 (local address 127.23.186.254:33769)
0811 14:30:56.893166 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:56.893178 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:56.893223 (+ 45us) server_negotiation.cc:407] Connection header received
0811 14:30:56.893778 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:56.893787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:56.893852 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:56.894019 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:56.894666 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:56.896140 (+ 1474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:56.897085 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:56.897585 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:56.897716 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:56.900637 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:56.900641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:56.900642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:56.900654 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:56.902830 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:56.903274 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:56.903278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:56.903278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:56.903395 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:56.903625 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:56.903626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:56.903627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:56.903938 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:56.904086 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:56.904578 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 14:30:56.904760 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":91,"threads_started":1}
I20260811 14:30:56.910416 1509 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:30:56.910964 1509 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:56.917594 1509 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Generated new TSK 0
I20260811 14:30:56.917824 1509 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:30:56.954262 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33769
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:57.053766 1523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:57.054049 1523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:57.054095 1523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:57.055413 1523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:57.055488 1523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:57.055552 1523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:57.057255 1523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:57.057571 1523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:57.057936 1523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:30:57.064216 1523 server_base.cc:1034] running on GCE node
W20260811 14:30:57.064221 1528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.064241 1529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.064221 1531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.064780 1523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:57.065018 1523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:57.066166 1523 hybrid_clock.cc:648] HybridClock initialized: now 1786458657066153 us; error 32 us; skew 500 ppm
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:57.068975 1523 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:30:57.069415 1523 webserver.cc:460] Webserver started at http://127.23.186.193:35639/ using document root <none> and password file <none>
I20260811 14:30:57.069633 1523 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:57.069676 1523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:57.069772 1523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:57.070834 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "592d141fa4e3445a8f1033f9af4ea27e"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "2284d109de049e23b951c8bcd84941e9"
server_key_iv: "9db03fcaea17405d9b50ff6dbb5f921a"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.071153 1523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "592d141fa4e3445a8f1033f9af4ea27e"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "2284d109de049e23b951c8bcd84941e9"
server_key_iv: "9db03fcaea17405d9b50ff6dbb5f921a"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.072502 1523 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:30:57.073390 1540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.073560 1523 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:57.073657 1523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "592d141fa4e3445a8f1033f9af4ea27e"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "2284d109de049e23b951c8bcd84941e9"
server_key_iv: "9db03fcaea17405d9b50ff6dbb5f921a"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.073720 1523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:57.101545 1523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:57.103996 1523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:57.104125 1523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:57.104328 1523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:57.104677 1523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:57.104736 1523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.104791 1523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:57.104817 1523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.110802 1523 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:43425
I20260811 14:30:57.110869 1675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:43425 every 8 connection(s)
I20260811 14:30:57.111235 1523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:57.120970 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 1523
I20260811 14:30:57.121111 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:30:57.121240 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.112120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:38669 (local address 127.23.186.254:33769)
0811 14:30:57.112214 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.112217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.112413 (+ 196us) server_negotiation.cc:407] Connection header received
0811 14:30:57.113019 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.113020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.113032 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.113087 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.114128 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.114654 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.115693 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.115901 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.115956 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.118427 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:57.118429 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:57.118429 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:57.118444 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:57.119718 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:57.120157 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:57.120159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:57.120159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:57.120223 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:57.120504 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:57.120506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:57.120506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:57.120633 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:57.120732 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:57.121030 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 14:30:57.121143 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 14:30:57.121327 24299 external_mini_cluster.cc:1428] Setting key 08aefb23f42eb409937be296f2636bc3
I20260811 14:30:57.121428 1678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.112066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:38669)
0811 14:30:57.112275 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.112317 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.112906 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.113178 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.113181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.113802 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.114028 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.114033 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.114750 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.114751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.115606 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.115608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.115983 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.115984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.116100 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.116630 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.116639 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.118317 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:57.119814 (+ 1497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:57.119824 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:57.119827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:57.120072 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:57.120295 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:57.120296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:57.120296 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:57.120418 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:57.120726 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:57.120734 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:57.120885 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 14:30:57.121008 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 14:30:57.121181 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260811 14:30:57.121985 1676 heartbeater.cc:347] Connected to a master server at 127.23.186.254:33769
I20260811 14:30:57.122134 1676 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:57.122346 1676 heartbeater.cc:511] Master 127.23.186.254:33769 requested a full tablet report, sending...
I20260811 14:30:57.122926 1450 ts_manager.cc:194] Registered new tserver with Master: 592d141fa4e3445a8f1033f9af4ea27e (127.23.186.193:43425)
I20260811 14:30:57.123582 1450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:38669
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:30:57.151381 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33769
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:57.236055 1682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:57.236334 1682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:57.236375 1682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:57.237711 1682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:57.237792 1682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:57.237838 1682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:57.239427 1682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:57.239728 1682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:57.239976 1682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:57.245664 1690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.245682 1688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.245684 1687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.245846 1682 server_base.cc:1034] running on GCE node
I20260811 14:30:57.246191 1682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:57.246512 1682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:57.247670 1682 hybrid_clock.cc:648] HybridClock initialized: now 1786458657247639 us; error 48 us; skew 500 ppm
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:57.250526 1682 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:30:57.251102 1682 webserver.cc:460] Webserver started at http://127.23.186.194:41963/ using document root <none> and password file <none>
I20260811 14:30:57.251368 1682 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:57.251447 1682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:57.251613 1682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:57.252843 1682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "09de677c923841469050a5f8493901a8"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "f1426bf9c0c2ae567c6b55b6853770e0"
server_key_iv: "12b421744b36dc57985791e04cc6f8a8"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.253265 1682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "09de677c923841469050a5f8493901a8"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "f1426bf9c0c2ae567c6b55b6853770e0"
server_key_iv: "12b421744b36dc57985791e04cc6f8a8"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.254804 1682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:30:57.255702 1700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.255918 1682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:57.256042 1682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "09de677c923841469050a5f8493901a8"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "f1426bf9c0c2ae567c6b55b6853770e0"
server_key_iv: "12b421744b36dc57985791e04cc6f8a8"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.256127 1682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:57.270782 1682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:57.273057 1682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:57.273252 1682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:57.273532 1682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:57.273916 1682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:57.273985 1682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.274025 1682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:57.274077 1682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.281155 1682 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:32917
I20260811 14:30:57.281210 1832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:32917 every 8 connection(s)
I20260811 14:30:57.281694 1682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:57.287592 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 1682
I20260811 14:30:57.287722 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:30:57.287921 24299 external_mini_cluster.cc:1428] Setting key db6841d3eae8847c56417f9caf1d5aca
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:57.292361 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.282538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:59797 (local address 127.23.186.254:33769)
0811 14:30:57.282618 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.282620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.282886 (+ 266us) server_negotiation.cc:407] Connection header received
0811 14:30:57.283373 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.283375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.283387 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.283454 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.284505 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.285145 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.286235 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.286454 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.286522 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.289269 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:57.289270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:57.289271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:57.289280 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:57.290901 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:57.291328 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:57.291330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:57.291330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:57.291396 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:57.291673 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:57.291674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:57.291674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:57.291801 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:57.291879 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:57.292174 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 14:30:57.292264 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 14:30:57.292578 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.282520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:59797)
0811 14:30:57.282711 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.282757 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.283262 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.283549 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.283552 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.284175 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.284382 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.284386 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.285263 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.285264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.286137 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.286140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.286541 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.286541 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.286661 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.287226 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.287235 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.289164 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:57.290996 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:57.291005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:57.291008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:57.291249 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:57.291480 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:57.291481 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:57.291482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:57.291598 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:57.291869 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:57.291876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:57.292032 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 14:30:57.292142 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 14:30:57.292322 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260811 14:30:57.293110 1833 heartbeater.cc:347] Connected to a master server at 127.23.186.254:33769
I20260811 14:30:57.293234 1833 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:57.293442 1833 heartbeater.cc:511] Master 127.23.186.254:33769 requested a full tablet report, sending...
I20260811 14:30:57.293874 1450 ts_manager.cc:194] Registered new tserver with Master: 09de677c923841469050a5f8493901a8 (127.23.186.194:32917)
I20260811 14:30:57.294274 1450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:59797
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:30:57.316388 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33769
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:57.402269 1840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:57.402535 1840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:57.402587 1840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:57.403823 1840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:57.403901 1840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:57.403950 1840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:57.405771 1840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:57.406073 1840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:57.406328 1840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:57.411702 1848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.411675 1845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.411725 1840 server_base.cc:1034] running on GCE node
W20260811 14:30:57.411685 1846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.412207 1840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:57.412464 1840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:57.413661 1840 hybrid_clock.cc:648] HybridClock initialized: now 1786458657413631 us; error 48 us; skew 500 ppm
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:57.416432 1840 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:30:57.417012 1840 webserver.cc:460] Webserver started at http://127.23.186.195:44009/ using document root <none> and password file <none>
I20260811 14:30:57.417300 1840 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:57.417373 1840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:57.417539 1840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:30:57.418823 1840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "1293be368fe84cda887828813c2ea8b9"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "3bb5b22e89ba879597e7d5abdf0f0935"
server_key_iv: "b276f336f94414260c832dd7380ba7d0"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.419224 1840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "1293be368fe84cda887828813c2ea8b9"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "3bb5b22e89ba879597e7d5abdf0f0935"
server_key_iv: "b276f336f94414260c832dd7380ba7d0"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.420789 1840 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:30:57.421742 1857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.421929 1840 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:57.422052 1840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "1293be368fe84cda887828813c2ea8b9"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "3bb5b22e89ba879597e7d5abdf0f0935"
server_key_iv: "b276f336f94414260c832dd7380ba7d0"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.422137 1840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:57.438354 1840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:57.441274 1840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:57.441463 1840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:57.441754 1840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:57.442088 1840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:57.442143 1840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.442189 1840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:57.442214 1840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.449098 1840 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:33957
I20260811 14:30:57.449157 1995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:33957 every 8 connection(s)
I20260811 14:30:57.449506 1840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:30:57.453316 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 1840
I20260811 14:30:57.453430 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:30:57.453613 24299 external_mini_cluster.cc:1428] Setting key 119f9804a390adbfbdcdff81f525231f
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:30:57.462039 1497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.450355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:57119 (local address 127.23.186.254:33769)
0811 14:30:57.450433 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.450435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.450633 (+ 198us) server_negotiation.cc:407] Connection header received
0811 14:30:57.451191 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.451193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.451206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.451273 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.452315 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.453233 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.454846 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.455204 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.455297 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.458055 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:57.458057 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:57.458057 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:57.458069 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:57.459982 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:57.460663 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:57.460665 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:57.460666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:57.460779 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:57.461052 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:57.461053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:57.461053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:57.461363 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:57.461460 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:57.461796 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 14:30:57.461901 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:30:57.462148 1998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.450301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:57119)
0811 14:30:57.450487 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.450544 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.451078 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.451378 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.451382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.451998 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.452209 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.452213 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.453344 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.453346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.454755 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.454759 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.455298 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.455299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.455427 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.456060 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.456070 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.457929 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:30:57.460186 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:57.460220 (+ 34us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:57.460223 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:57.460572 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:57.460863 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:30:57.460864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:30:57.460865 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:30:57.460985 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:30:57.461473 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:30:57.461481 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:30:57.461635 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 14:30:57.461747 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:30:57.461916 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
I20260811 14:30:57.462687 1996 heartbeater.cc:347] Connected to a master server at 127.23.186.254:33769
I20260811 14:30:57.462821 1996 heartbeater.cc:464] Registering TS with master...
I20260811 14:30:57.463049 1996 heartbeater.cc:511] Master 127.23.186.254:33769 requested a full tablet report, sending...
I20260811 14:30:57.463593 1450 ts_manager.cc:194] Registered new tserver with Master: 1293be368fe84cda887828813c2ea8b9 (127.23.186.195:33957)
I20260811 14:30:57.464170 1450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:57119
I20260811 14:30:57.465607 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab.
Entry for principal oryx/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab.
WARNING: no policy specified for oryx/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab.
Entry for principal oryx/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab.
WARNING: no policy specified for oryx/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab.
Entry for principal oryx/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab.
I20260811 14:30:57.559001 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 1523
I20260811 14:30:57.567214 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 1682
I20260811 14:30:57.576933 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 1840
I20260811 14:30:57.582033 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 1409
I20260811 14:30:57.590694 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33769
--webserver_interface=127.23.186.254
--webserver_port=45481
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:33769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:30:57.676179 2008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:57.676397 2008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:57.676425 2008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:57.677634 2008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:30:57.677680 2008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:57.677695 2008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:57.677707 2008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:30:57.677719 2008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:30:57.679199 2008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:33769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=45481
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:57.679426 2008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:57.679610 2008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:57.684758 2015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.684775 2017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.684868 2008 server_base.cc:1034] running on GCE node
W20260811 14:30:57.684775 2013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.685218 2008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:57.685529 2008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:57.686708 2008 hybrid_clock.cc:648] HybridClock initialized: now 1786458657686683 us; error 36 us; skew 500 ppm
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:57.689409 2008 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:30:57.689951 2008 webserver.cc:460] Webserver started at http://127.23.186.254:45481/ using document root <none> and password file <none>
I20260811 14:30:57.690231 2008 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:57.690328 2008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:57.691895 2008 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260811 14:30:57.692749 2025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.692925 2008 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:57.693045 2008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "e257179d07234303b5116438d114182d"
format_stamp: "Formatted at 2026-08-11 14:30:56 on dist-test-slave-cbsf"
server_key: "34757b78e7a019db4548efb7f00914b2"
server_key_iv: "01b080220ced95cd4599f09289e1f2bf"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.693421 2008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:57.702203 2008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:57.704334 2008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:57.704519 2008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:57.708899 2008 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:33769
I20260811 14:30:57.708942 2093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:33769 every 8 connection(s)
I20260811 14:30:57.709339 2008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:30:57.710884 2094 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:30:57.711589 2094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Bootstrap starting.
I20260811 14:30:57.716902 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2008
I20260811 14:30:57.717303 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:43425
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:33769
--webserver_port=35639
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:30:57.720911 2094 log.cc:824] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Log is configured to *not* fsync() on all Append() calls
I20260811 14:30:57.722806 2094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:30:57.723135 2094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Bootstrap complete.
I20260811 14:30:57.725615 2094 raft_consensus.cc:348] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:57.725849 2094 raft_consensus.cc:729] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e257179d07234303b5116438d114182d, State: Initialized, Role: FOLLOWER
I20260811 14:30:57.726032 2094 consensus_queue.cc:260] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:57.726115 2094 raft_consensus.cc:388] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:30:57.726189 2094 raft_consensus.cc:482] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:30:57.726241 2094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:30:57.727298 2094 raft_consensus.cc:504] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:57.727391 2094 leader_election.cc:302] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e257179d07234303b5116438d114182d; no voters:
I20260811 14:30:57.727632 2094 leader_election.cc:288] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:30:57.727671 2100 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:30:57.727802 2100 raft_consensus.cc:686] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [term 2 LEADER]: Becoming Leader. State: Replica: e257179d07234303b5116438d114182d, State: Running, Role: LEADER
I20260811 14:30:57.727893 2100 consensus_queue.cc:237] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } }
I20260811 14:30:57.727929 2094 sys_catalog.cc:564] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:30:57.728197 2100 sys_catalog.cc:455] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e257179d07234303b5116438d114182d. Latest consensus state: current_term: 2 leader_uuid: "e257179d07234303b5116438d114182d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } } }
I20260811 14:30:57.728277 2100 sys_catalog.cc:458] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:57.728204 2102 sys_catalog.cc:455] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e257179d07234303b5116438d114182d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e257179d07234303b5116438d114182d" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33769 } } }
I20260811 14:30:57.728782 2102 sys_catalog.cc:458] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d [sys.catalog]: This master's current role is: LEADER
I20260811 14:30:57.730540 2117 catalog_manager.cc:1259] Loaded cluster ID: f2cebd6db2644b4785db85442b86d99a
I20260811 14:30:57.730598 2117 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: loading cluster ID for follower catalog manager: success
I20260811 14:30:57.731789 2117 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: acquiring CA information for follower catalog manager: success
I20260811 14:30:57.732183 2117 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:30:57.732263 2104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:30:57.732419 2104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:30:57.732491 2104 catalog_manager.cc:1259] Loaded cluster ID: f2cebd6db2644b4785db85442b86d99a
I20260811 14:30:57.732513 2104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:30:57.732733 2104 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:30:57.732812 2104 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e257179d07234303b5116438d114182d: Loaded TSK: 0
I20260811 14:30:57.732949 2104 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:30:57.805378 2096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:57.805603 2096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:57.805627 2096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:57.806867 2096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:57.806915 2096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:57.806944 2096 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:30:57.808442 2096 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.193.keytab
--principal=oryx/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:43425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=35639
--tserver_master_addrs=127.23.186.254:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:57.808701 2096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:57.809001 2096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:57.814265 2126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.814280 2123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.814452 2096 server_base.cc:1034] running on GCE node
W20260811 14:30:57.814541 2124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.814752 2096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:57.815017 2096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:57.816185 2096 hybrid_clock.cc:648] HybridClock initialized: now 1786458657816172 us; error 33 us; skew 500 ppm
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:57.819075 2096 init.cc:377] Logged in from keytab as oryx/127.23.186.193@KRBTEST.COM (short username oryx)
I20260811 14:30:57.819712 2096 webserver.cc:460] Webserver started at http://127.23.186.193:35639/ using document root <none> and password file <none>
I20260811 14:30:57.819944 2096 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:57.820042 2096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:57.821619 2096 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:30:57.822335 2134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.822522 2096 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:30:57.822645 2096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "592d141fa4e3445a8f1033f9af4ea27e"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "2284d109de049e23b951c8bcd84941e9"
server_key_iv: "9db03fcaea17405d9b50ff6dbb5f921a"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.823009 2096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:57.831534 2096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:57.834172 2096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:57.834358 2096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:57.834667 2096 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:57.835093 2096 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:57.835132 2096 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.835157 2096 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:57.835202 2096 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.842931 2096 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:43425
I20260811 14:30:57.842964 2270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:43425 every 8 connection(s)
I20260811 14:30:57.843395 2096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:30:57.843940 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2096
I20260811 14:30:57.844244 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:32917
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:33769
--webserver_port=41963
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:57.854666 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.846377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:52901)
0811 14:30:57.846716 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.846751 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.847290 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.847784 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.847788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.848417 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.848623 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.848629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.849889 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.849890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.850690 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.850693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.851322 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.851324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.851471 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.852041 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.852050 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.854421 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":102,"threads_started":1}
W20260811 14:30:57.854748 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:57.846412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:52901 (local address 127.23.186.254:33769)
0811 14:30:57.846626 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.846636 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.846872 (+ 236us) server_negotiation.cc:407] Connection header received
0811 14:30:57.847518 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.847524 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.847567 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.847746 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.848802 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.849761 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.850791 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.851221 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.851359 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.854576 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:52901: BlockingRecv error: recv got EOF from 127.23.186.193:52901 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260811 14:30:57.855407 2272 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:33769 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.186.254:33769: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:57.860280 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.855681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:43169)
0811 14:30:57.855792 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.855804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.855852 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.856008 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.856009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.856154 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.856204 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.856205 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.856918 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.856919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.857431 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.857433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.857805 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.857806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.857879 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.858342 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.858347 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.860190 (+ 1843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":96}
W20260811 14:30:57.860337 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:57.855740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:43169 (local address 127.23.186.254:33769)
0811 14:30:57.855796 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.855799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.855835 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:30:57.855914 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.855915 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.855926 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.856005 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.856293 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.856841 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.857539 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.857732 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.857807 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.860274 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:43169: BlockingRecv error: recv got EOF from 127.23.186.193:43169 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:57.865060 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.860829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:58829)
0811 14:30:57.860874 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.860884 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.860941 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.861063 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.861064 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.861195 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.861224 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.861225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.861852 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.861853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.862347 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.862349 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.862707 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.862708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.862759 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.863217 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.863222 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.864968 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:30:57.865118 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:57.860894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:58829 (local address 127.23.186.254:33769)
0811 14:30:57.860970 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.860972 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.860979 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:30:57.861002 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.861003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.861012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.861061 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.861305 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.861773 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.862434 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.862634 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.862709 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.865054 (+ 2345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:58829: BlockingRecv error: recv got EOF from 127.23.186.193:58829 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 14:30:57.865530 2272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:30:57.936583 2274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:57.936838 2274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:57.936864 2274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:57.938015 2274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:57.938054 2274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:57.938081 2274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:30:57.939494 2274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.194.keytab
--principal=oryx/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:32917
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=41963
--tserver_master_addrs=127.23.186.254:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:57.939708 2274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:57.939899 2274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:30:57.945042 2286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.945076 2281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:57.945103 2283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:57.945222 2274 server_base.cc:1034] running on GCE node
I20260811 14:30:57.945482 2274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:57.945746 2274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:57.946888 2274 hybrid_clock.cc:648] HybridClock initialized: now 1786458657946869 us; error 41 us; skew 500 ppm
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458657, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:57.949506 2274 init.cc:377] Logged in from keytab as oryx/127.23.186.194@KRBTEST.COM (short username oryx)
I20260811 14:30:57.949997 2274 webserver.cc:460] Webserver started at http://127.23.186.194:41963/ using document root <none> and password file <none>
I20260811 14:30:57.950238 2274 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:57.950325 2274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:57.951797 2274 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:57.952476 2293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.952678 2274 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:30:57.952840 2274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "09de677c923841469050a5f8493901a8"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "f1426bf9c0c2ae567c6b55b6853770e0"
server_key_iv: "12b421744b36dc57985791e04cc6f8a8"
server_key_version: "encryptionkey@0"
I20260811 14:30:57.953208 2274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:57.966495 2274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:57.968977 2274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:57.969154 2274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:57.969411 2274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:57.969733 2274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:57.969790 2274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.969835 2274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:57.969859 2274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:57.976019 2274 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:32917
I20260811 14:30:57.976074 2431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:32917 every 8 connection(s)
I20260811 14:30:57.976460 2274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:30:57.981132 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2274
I20260811 14:30:57.981468 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:33957
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:33769
--webserver_port=44009
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:42429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 14:30:57.985096 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:57.977336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:42743 (local address 127.23.186.254:33769)
0811 14:30:57.977409 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.977412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.977682 (+ 270us) server_negotiation.cc:407] Connection header received
0811 14:30:57.978174 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.978175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.978189 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.978244 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.979292 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.979804 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.981000 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.981314 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.981405 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.985045 (+ 3640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:42743: BlockingRecv error: recv got EOF from 127.23.186.194:42743 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:30:57.985179 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.977330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:42743)
0811 14:30:57.977543 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.977583 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.978086 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.978336 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.978339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.978986 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.979198 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.979203 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.979918 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.979919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.980897 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.980900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.981432 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.981433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.981556 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.982219 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.982230 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.984935 (+ 2705us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":118,"threads_started":1}
W20260811 14:30:57.985417 2432 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:33769 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.186.254:33769: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:57.990270 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.985694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:45535)
0811 14:30:57.985758 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.985774 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.985840 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.985988 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.985989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.986153 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.986208 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.986209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.986858 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.986859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.987365 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.987366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.987727 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.987728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.987800 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.988312 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.988316 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.990219 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 14:30:57.990429 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:57.985779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:45535 (local address 127.23.186.254:33769)
0811 14:30:57.985849 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.985852 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.985862 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:30:57.985903 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.985904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.985921 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.985983 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.986291 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.986774 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.987453 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.987652 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.987729 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.990351 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:45535: BlockingRecv error: recv got EOF from 127.23.186.194:45535 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:57 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:57.995064 2434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:57.990585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:34775)
0811 14:30:57.990684 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:57.990692 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:57.990723 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:57.990891 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:57.990892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:57.991017 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:57.991046 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.991047 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.991792 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.991793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.992268 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:57.992270 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.992628 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.992628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:57.992725 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.993205 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:57.993209 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:57.994984 (+ 1775us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":83}
W20260811 14:30:57.995118 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:57.990707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:34775 (local address 127.23.186.254:33769)
0811 14:30:57.990773 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:57.990776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:57.990784 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:57.990806 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:57.990806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:57.990816 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:57.990887 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:57.991133 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.991702 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.992349 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:57.992547 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:57.992624 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:57.995056 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:34775: BlockingRecv error: recv got EOF from 127.23.186.194:34775 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 14:30:57.995214 2432 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:30:58.071501 2435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:30:58.071727 2435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:30:58.071753 2435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:30:58.073041 2435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:30:58.073089 2435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:30:58.073118 2435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:30:58.074666 2435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:42429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/krb5kdc/oryx_127.23.186.195.keytab
--principal=oryx/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:33957
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=44009
--tserver_master_addrs=127.23.186.254:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:30:58.074939 2435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:30:58.075189 2435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:30:58.080343 2435 server_base.cc:1034] running on GCE node
W20260811 14:30:58.080257 2440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:58.080278 2443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:30:58.080295 2441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:30:58.080775 2435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:30:58.081054 2435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:30:58.082248 2435 hybrid_clock.cc:648] HybridClock initialized: now 1786458658082245 us; error 42 us; skew 500 ppm
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458658, etypes {rep=17 tkt=17 ses=17}, oryx/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:30:58.084883 2435 init.cc:377] Logged in from keytab as oryx/127.23.186.195@KRBTEST.COM (short username oryx)
I20260811 14:30:58.085354 2435 webserver.cc:460] Webserver started at http://127.23.186.195:44009/ using document root <none> and password file <none>
I20260811 14:30:58.085621 2435 fs_manager.cc:359] Metadata directory not provided
I20260811 14:30:58.085709 2435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:30:58.087085 2435 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:30:58.087674 2452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:58.087960 2435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:30:58.088070 2435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "1293be368fe84cda887828813c2ea8b9"
format_stamp: "Formatted at 2026-08-11 14:30:57 on dist-test-slave-cbsf"
server_key: "3bb5b22e89ba879597e7d5abdf0f0935"
server_key_iv: "b276f336f94414260c832dd7380ba7d0"
server_key_version: "encryptionkey@0"
I20260811 14:30:58.088495 2435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:30:58.102628 2435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:30:58.104660 2435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:30:58.104914 2435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:30:58.105180 2435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:30:58.105547 2435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:30:58.105603 2435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:58.105638 2435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:30:58.105669 2435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:30:58.111778 2435 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:33957
I20260811 14:30:58.111851 2585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:33957 every 8 connection(s)
I20260811 14:30:58.112247 2435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:58.118750 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2435
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:58.119853 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:58.113099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:38513)
0811 14:30:58.113284 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:58.113322 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:58.113803 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:58.114007 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:58.114010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:58.114630 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:58.114834 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.114838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.115482 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.115483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.116289 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.116291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.116619 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.116620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.116752 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.117352 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:58.117361 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:58.119703 (+ 2342us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":99,"threads_started":1}
W20260811 14:30:58.119887 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:58.113099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:38513 (local address 127.23.186.254:33769)
0811 14:30:58.113189 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:58.113191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:58.113413 (+ 222us) server_negotiation.cc:407] Connection header received
0811 14:30:58.113887 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:58.113888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:58.113899 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:58.113954 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:58.114911 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.115404 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.116368 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.116547 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.116597 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.119806 (+ 3209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:38513: BlockingRecv error: recv got EOF from 127.23.186.195:38513 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 14:30:58.120141 2586 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:33769 (0 consecutive failures): Not authorized: Failed to ping master at 127.23.186.254:33769: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:58.125468 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:58.120383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:56775)
0811 14:30:58.120429 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:58.120439 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:58.120475 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:58.120599 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:58.120600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:58.120776 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:58.120827 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.120829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.121621 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.121623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.122451 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.122454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.122898 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.122899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.122985 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.123407 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:58.123411 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:58.125380 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:30:58.125535 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:58.120425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:56775 (local address 127.23.186.254:33769)
0811 14:30:58.120460 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:58.120462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:58.120470 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:30:58.120535 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:58.120536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:58.120543 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:58.120591 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:58.120887 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.121540 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.122533 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.122823 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.122891 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.125455 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:56775: BlockingRecv error: recv got EOF from 127.23.186.195:56775 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 14:30:58.128192 2590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:58.119310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.23.186.254:33769)
0811 14:30:58.119451 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:58.119453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:58.119465 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:30:58.119495 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:58.119496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:58.119508 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:58.119570 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:58.120745 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.121530 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.122606 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.122890 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.122949 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.124390 (+ 1441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:30:58.124396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:30:58.124397 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:30:58.124413 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:30:58.126743 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:58.127126 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:58.127129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:58.127129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:58.127226 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:30:58.127432 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:30:58.127433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:30:58.127433 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:30:58.127662 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:30:58.127795 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 14:30:58.127950 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 14:30:58.128065 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:58.130479 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:58.125795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:48949)
0811 14:30:58.125840 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:58.125849 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:58.125883 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:58.126026 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:58.126027 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:58.126128 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:58.126157 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.126159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.127038 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.127039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.127684 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.127686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.128157 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.128159 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.128217 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.128619 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:58.128624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:58.130433 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 14:30:58.130643 2586 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:30:58.130553 2276 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:58.125838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:48949 (local address 127.23.186.254:33769)
0811 14:30:58.125906 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:58.125908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:58.125917 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:30:58.125938 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:58.125939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:58.125948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:58.126018 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:58.126252 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.126962 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.127778 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.128086 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.128156 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.130514 (+ 2358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:48949: BlockingRecv error: recv got EOF from 127.23.186.195:48949 (error 108)
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":39}
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:58.871436 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:58.866091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:47219)
0811 14:30:58.866290 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:58.866303 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:58.866352 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:58.866493 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:58.866494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:58.866688 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:58.866741 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.866743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.867500 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.867501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.868066 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.868068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.868439 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.868441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.868502 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.869163 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:58.869169 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:58.871285 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":124,"threads_started":1}
W20260811 14:30:58.871476 2592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:58.866183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:47219 (local address 127.23.186.254:33769)
0811 14:30:58.866325 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:58.866327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:58.866370 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:30:58.866401 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:58.866401 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:58.866411 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:58.866465 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:58.866850 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.867409 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.868148 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.868364 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.868413 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.871392 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:47219: BlockingRecv error: recv got EOF from 127.23.186.193:47219 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:58 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:59.002030 2593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:58.995682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:53139)
0811 14:30:58.995957 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:58.995971 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:58.996060 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:58.996204 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:58.996205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:58.996429 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:58.996496 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.996498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.997560 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.997561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.998409 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:58.998413 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.998897 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.998897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:58.998977 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:58.999557 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:58.999564 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:59.001962 (+ 2398us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":76,"threads_started":1}
W20260811 14:30:59.002099 2592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:58.995731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:53139 (local address 127.23.186.254:33769)
0811 14:30:58.995825 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:58.995827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:58.996047 (+ 220us) server_negotiation.cc:407] Connection header received
0811 14:30:58.996135 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:58.996136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:58.996149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:58.996201 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:58.996582 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.997468 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.998505 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:58.998812 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:58.998892 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:59.002051 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:53139: BlockingRecv error: recv got EOF from 127.23.186.194:53139 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 14:30:59 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:59 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:59.136077 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:59.131125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:60765)
0811 14:30:59.131302 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:59.131318 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:59.131381 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:59.131526 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:59.131527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:59.131706 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:59.131760 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:59.131762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.132443 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.132444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:59.133110 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:59.133113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.133572 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.133573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:59.133660 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:59.134194 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:59.134206 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:59.136004 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":119,"threads_started":1}
W20260811 14:30:59.136215 2592 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:59.131195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:60765 (local address 127.23.186.254:33769)
0811 14:30:59.131277 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:59.131279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:59.131384 (+ 105us) server_negotiation.cc:407] Connection header received
0811 14:30:59.131429 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:59.131430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:59.131441 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:59.131504 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:59.131837 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.132353 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.133241 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.133471 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.133563 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:59.136129 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:60765: BlockingRecv error: recv got EOF from 127.23.186.195:60765 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 14:30:59 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:30:59 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:30:59.877347 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:30:59.872151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:40237)
0811 14:30:59.872354 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 14:30:59.872367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:30:59.872437 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:30:59.872590 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:30:59.872591 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:30:59.872857 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:30:59.872922 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:59.872924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.873837 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.873838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:59.874361 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:30:59.874362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.874697 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.874698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:30:59.874740 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:59.875260 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:30:59.875266 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:30:59.877269 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":110,"threads_started":1}
W20260811 14:30:59.877465 2596 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:30:59.872282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:40237 (local address 127.23.186.254:33769)
0811 14:30:59.872417 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 14:30:59.872419 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:30:59.872430 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:30:59.872490 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:30:59.872490 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:30:59.872500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:30:59.872572 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:30:59.873024 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.873767 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.874435 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:30:59.874634 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:30:59.874683 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:30:59.877376 (+ 2693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:40237: BlockingRecv error: recv got EOF from 127.23.186.193:40237 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":80,"threads_started":1}
Aug 11 14:31:00 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:00 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:00.007925 2597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:00.002658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:55237)
0811 14:31:00.002926 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:00.002939 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:00.003051 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:00.003226 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:00.003227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:00.003432 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:00.003482 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:00.003484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.004202 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.004203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:00.004770 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:00.004772 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.005163 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.005164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:00.005231 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:00.005773 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:00.005778 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:00.007794 (+ 2016us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":143,"threads_started":1}
W20260811 14:31:00.008008 2596 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:00.002785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:55237 (local address 127.23.186.254:33769)
0811 14:31:00.002924 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:00.002926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:00.003030 (+ 104us) server_negotiation.cc:407] Connection header received
0811 14:31:00.003135 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:00.003136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:00.003150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:00.003202 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:00.003574 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.004102 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.004860 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.005073 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.005161 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:00.007926 (+ 2765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:55237: BlockingRecv error: recv got EOF from 127.23.186.194:55237 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 14:31:00 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:00 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:00.141659 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:00.136643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:41417)
0811 14:31:00.136831 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:00.136845 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:00.136938 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:00.137102 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:00.137103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:00.137305 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:00.137357 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:00.137358 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.138058 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.138059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:00.138593 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:00.138594 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.139039 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.139040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:00.139111 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:00.139648 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:00.139653 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:00.141584 (+ 1931us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":118,"threads_started":1}
W20260811 14:31:00.141753 2596 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:00.136873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:41417 (local address 127.23.186.254:33769)
0811 14:31:00.136946 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:00.136948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:00.136977 (+ 29us) server_negotiation.cc:407] Connection header received
0811 14:31:00.137006 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:00.137007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:00.137020 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:00.137095 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:00.137447 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.137960 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.138688 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.138959 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.139038 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:00.141686 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:41417: BlockingRecv error: recv got EOF from 127.23.186.195:41417 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 14:31:00 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:00 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:00.883195 2599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:00.878031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:51041)
0811 14:31:00.878270 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:00.878284 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:00.878370 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:00.878545 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:00.878546 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:00.878737 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:00.878795 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:00.878797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.879507 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.879508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:00.880050 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:00.880052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.880441 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.880442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:00.880490 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:00.881091 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:00.881097 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:00.883105 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":135,"threads_started":1}
W20260811 14:31:00.883288 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:00.878143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:51041 (local address 127.23.186.254:33769)
0811 14:31:00.878263 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:00.878265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:00.878368 (+ 103us) server_negotiation.cc:407] Connection header received
0811 14:31:00.878440 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:00.878441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:00.878452 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:00.878515 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:00.878876 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.879418 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.880135 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:00.880329 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:00.880431 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:00.883215 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:51041: BlockingRecv error: recv got EOF from 127.23.186.193:51041 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
Aug 11 14:31:01 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:01 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:01.013701 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:01.008673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:49651)
0811 14:31:01.008862 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:01.008878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:01.008961 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:01.009110 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:01.009111 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:01.009290 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:01.009340 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:01.009342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.010004 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.010005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:01.010545 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:01.010547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.010894 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.010896 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:01.010939 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:01.011425 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:01.011431 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:01.013629 (+ 2198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":100,"threads_started":1}
W20260811 14:31:01.013832 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:01.008834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:49651 (local address 127.23.186.254:33769)
0811 14:31:01.008892 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:01.008895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:01.008944 (+ 49us) server_negotiation.cc:407] Connection header received
0811 14:31:01.009031 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:01.009033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:01.009046 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:01.009087 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:01.009411 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.009920 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.010636 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.010823 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.010878 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:01.013729 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:49651: BlockingRecv error: recv got EOF from 127.23.186.194:49651 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 11 14:31:01 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:01 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:01.147270 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:01.142364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:55161)
0811 14:31:01.142530 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:01.142541 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:01.142604 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:01.142758 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:01.142759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:01.142942 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:01.142996 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:01.142998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.143672 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.143673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:01.144294 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:01.144297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.144779 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.144780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:01.144867 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:01.145331 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:01.145336 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:01.147207 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":82,"threads_started":1}
W20260811 14:31:01.147430 2601 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:01.142459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:55161 (local address 127.23.186.254:33769)
0811 14:31:01.142549 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:01.142551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:01.142593 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:31:01.142655 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:01.142655 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:01.142670 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:01.142737 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:01.143080 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.143590 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.144376 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.144668 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.144775 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:01.147356 (+ 2581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:55161: BlockingRecv error: recv got EOF from 127.23.186.195:55161 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 14:31:01 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:01 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:01.896908 2604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:01.883900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:57471)
0811 14:31:01.884128 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:01.884143 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:01.884254 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:01.884487 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:01.884488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:01.884752 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:01.884814 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:01.884816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.885485 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.885487 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:01.886057 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:01.886060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.886457 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.886458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:01.886500 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:01.894388 (+ 7888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:01.894396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:01.896844 (+ 2448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":128,"threads_started":1}
W20260811 14:31:01.897075 2605 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:01.884059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:57471 (local address 127.23.186.254:33769)
0811 14:31:01.884192 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:01.884194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:01.884232 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:31:01.884368 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:01.884370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:01.884384 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:01.884457 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:01.884889 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.885400 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.886170 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:01.886370 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:01.886435 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:01.896958 (+ 10523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:57471: BlockingRecv error: recv got EOF from 127.23.186.193:57471 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
Aug 11 14:31:02 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:02 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:02.019361 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:02.014271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:39815)
0811 14:31:02.014432 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:02.014449 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:02.014510 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:02.014677 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:02.014678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:02.014855 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:02.014908 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:02.014910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.015631 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.015633 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:02.016193 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:02.016194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.016615 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.016616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:02.016685 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:02.017205 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:02.017213 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:02.019263 (+ 2050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":101,"threads_started":1}
W20260811 14:31:02.019429 2605 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:02.014385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:39815 (local address 127.23.186.254:33769)
0811 14:31:02.014460 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:02.014462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:02.014496 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:31:02.014576 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:02.014577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:02.014589 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:02.014656 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:02.014978 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.015504 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.016299 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.016507 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.016613 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:02.019364 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:39815: BlockingRecv error: recv got EOF from 127.23.186.194:39815 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 14:31:02 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:02 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:02.153288 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:02.147809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:38233)
0811 14:31:02.148033 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:02.148050 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:02.148145 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:02.148327 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:02.148329 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:02.148532 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:02.148587 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:02.148589 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.149335 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.149337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:02.150015 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:02.150017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.150543 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.150545 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:02.150645 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:02.151234 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:02.151239 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:02.153224 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
W20260811 14:31:02.153506 2605 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:02.147960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:38233 (local address 127.23.186.254:33769)
0811 14:31:02.148068 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:02.148071 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:02.148120 (+ 49us) server_negotiation.cc:407] Connection header received
0811 14:31:02.148223 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:02.148224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:02.148240 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:02.148318 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:02.148710 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.149229 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.150117 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.150465 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.150542 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:02.153370 (+ 2828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:38233: BlockingRecv error: recv got EOF from 127.23.186.195:38233 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 14:31:02 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:02 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:02.902669 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:02.897464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:54077)
0811 14:31:02.897637 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:02.897647 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:02.897704 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:02.897898 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:02.897899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:02.898089 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:02.898147 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:02.898149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.898877 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.898878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:02.899459 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:02.899461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.899822 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.899823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:02.899868 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:02.900465 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:02.900471 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:02.902542 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
W20260811 14:31:02.902770 2609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:02.897550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:54077 (local address 127.23.186.254:33769)
0811 14:31:02.897688 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:02.897690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:02.897699 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:31:02.897764 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:02.897765 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:02.897776 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:02.897840 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:02.898264 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.898793 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.899546 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:02.899747 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:02.899804 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:02.902674 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:54077: BlockingRecv error: recv got EOF from 127.23.186.193:54077 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
Aug 11 14:31:03 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:03 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:03.025096 2610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:03.020034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:56065)
0811 14:31:03.020225 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:03.020236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:03.020290 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:03.020459 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:03.020460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:03.020635 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:03.020744 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:03.020746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.021439 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.021441 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:03.021985 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:03.021987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.022358 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.022359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:03.022410 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:03.023004 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:03.023009 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:03.025031 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
W20260811 14:31:03.025199 2609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:03.020201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:56065 (local address 127.23.186.254:33769)
0811 14:31:03.020258 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:03.020261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:03.020274 (+ 13us) server_negotiation.cc:407] Connection header received
0811 14:31:03.020373 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:03.020374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:03.020384 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:03.020431 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:03.020824 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.021341 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.022082 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.022282 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.022334 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:03.025132 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:56065: BlockingRecv error: recv got EOF from 127.23.186.194:56065 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 14:31:03 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:03 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:03.158830 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:03.153894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:41693)
0811 14:31:03.154111 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:03.154123 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:03.154193 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:03.154328 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:03.154329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:03.154496 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:03.154564 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:03.154565 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.155201 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.155201 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:03.155734 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:03.155735 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.156196 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.156197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:03.156278 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:03.156912 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:03.156917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:03.158769 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":128,"threads_started":1}
W20260811 14:31:03.158938 2609 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:03.154056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:41693 (local address 127.23.186.254:33769)
0811 14:31:03.154107 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:03.154109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:03.154192 (+ 83us) server_negotiation.cc:407] Connection header received
0811 14:31:03.154242 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:03.154243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:03.154256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:03.154307 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:03.154628 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.155132 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.155845 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.156124 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.156197 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:03.158868 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:41693: BlockingRecv error: recv got EOF from 127.23.186.195:41693 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 11 14:31:03 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:03 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:03.908509 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:03.903426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:36971)
0811 14:31:03.903617 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:03.903630 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:03.903691 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:03.903834 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:03.903834 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:03.904025 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:03.904078 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:03.904080 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.904806 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.904807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:03.905379 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:03.905381 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.905778 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.905779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:03.905821 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:03.906317 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:03.906322 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:03.908407 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":115,"threads_started":1}
W20260811 14:31:03.908604 2614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:03.903542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:36971 (local address 127.23.186.254:33769)
0811 14:31:03.903703 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:03.903705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:03.903714 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:31:03.903741 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:03.903742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:03.903752 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:03.903813 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:03.904152 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.904667 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.905484 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:03.905701 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:03.905766 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:03.908512 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:36971: BlockingRecv error: recv got EOF from 127.23.186.193:36971 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
Aug 11 14:31:04 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:04 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:04.031656 2615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:04.025855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:37861)
0811 14:31:04.026103 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:04.026122 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:04.026191 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:04.026378 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:04.026379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:04.026558 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:04.026608 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:04.026609 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.027470 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.027472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:04.028279 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:04.028283 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.028683 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.028684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:04.028799 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:04.029285 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:04.029291 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:04.031555 (+ 2264us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":127,"threads_started":1}
W20260811 14:31:04.031733 2614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:04.026066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:37861 (local address 127.23.186.254:33769)
0811 14:31:04.026130 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:04.026133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:04.026175 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:31:04.026262 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:04.026263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:04.026277 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:04.026345 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:04.026726 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.027370 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.028370 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.028592 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.028661 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:04.031662 (+ 3001us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:37861: BlockingRecv error: recv got EOF from 127.23.186.194:37861 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 11 14:31:04 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:04 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:04.164225 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:04.159408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:51501)
0811 14:31:04.159604 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:04.159615 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:04.159676 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:04.159831 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:04.159832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:04.160006 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:04.160064 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:04.160066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.160783 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.160784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:04.161322 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:04.161324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.161712 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.161713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:04.161781 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:04.162323 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:04.162328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:04.164161 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":124,"threads_started":1}
W20260811 14:31:04.164347 2614 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:04.159508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:51501 (local address 127.23.186.254:33769)
0811 14:31:04.159604 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:04.159606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:04.159679 (+ 73us) server_negotiation.cc:407] Connection header received
0811 14:31:04.159747 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:04.159748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:04.159762 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:04.159808 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:04.160131 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.160635 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.161402 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.161596 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.161653 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:04.164265 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:51501: BlockingRecv error: recv got EOF from 127.23.186.195:51501 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 14:31:04 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:04 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:04.914049 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:04.909083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:47777)
0811 14:31:04.909253 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:04.909268 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:04.909349 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:04.909499 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:04.909500 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:04.909682 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:04.909733 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:04.909735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.910398 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.910399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:04.910940 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:04.910942 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.911307 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.911309 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:04.911355 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:04.911922 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:04.911927 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:04.913973 (+ 2046us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
W20260811 14:31:04.914177 2618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:04.909205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:47777 (local address 127.23.186.254:33769)
0811 14:31:04.909346 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:04.909348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:04.909358 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:04.909403 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:04.909403 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:04.909415 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:04.909473 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:04.909801 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.910317 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.911025 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:04.911225 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:04.911279 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:04.914091 (+ 2812us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:47777: BlockingRecv error: recv got EOF from 127.23.186.193:47777 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":75,"threads_started":1}
Aug 11 14:31:05 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:05 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:05.037843 2619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:05.032474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:57527)
0811 14:31:05.032684 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:05.032699 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:05.032833 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:05.033025 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:05.033026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:05.033230 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:05.033282 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:05.033283 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.034001 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.034002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:05.034547 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:05.034548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.035031 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.035033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:05.035103 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:05.035633 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:05.035638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:05.037781 (+ 2143us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":114,"threads_started":1}
W20260811 14:31:05.038012 2618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:05.032606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:57527 (local address 127.23.186.254:33769)
0811 14:31:05.032724 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:05.032727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:05.032804 (+ 77us) server_negotiation.cc:407] Connection header received
0811 14:31:05.032922 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:05.032923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:05.032936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:05.033020 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:05.033377 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.033896 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.034668 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.034936 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.035017 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:05.037921 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:57527: BlockingRecv error: recv got EOF from 127.23.186.194:57527 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 14:31:05 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:05 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:05.170097 2620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:05.164843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:46799)
0811 14:31:05.165044 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:05.165060 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:05.165135 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:05.165298 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:05.165299 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:05.165511 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:05.165564 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:05.165566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.166296 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.166297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:05.166837 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:05.166839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.167228 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.167229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:05.167296 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:05.167810 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:05.167815 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:05.169989 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":102,"threads_started":1}
W20260811 14:31:05.170169 2618 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:05.164989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:46799 (local address 127.23.186.254:33769)
0811 14:31:05.165060 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:05.165063 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:05.165126 (+ 63us) server_negotiation.cc:407] Connection header received
0811 14:31:05.165195 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:05.165196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:05.165211 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:05.165287 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:05.165653 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.166184 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.166934 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.167134 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.167216 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:05.170101 (+ 2885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:46799: BlockingRecv error: recv got EOF from 127.23.186.195:46799 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 14:31:05 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:05 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:05.919560 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:05.914663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:45049)
0811 14:31:05.914853 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:05.914866 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:05.914937 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:05.915094 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:05.915095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:05.915281 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:05.915335 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:05.915337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.916000 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.916000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:05.916530 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:05.916531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.916993 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.916994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:05.917039 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:05.917559 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:05.917564 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:05.919494 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
W20260811 14:31:05.919770 2623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:05.914806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:45049 (local address 127.23.186.254:33769)
0811 14:31:05.914949 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:05.914951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:05.914961 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:05.914996 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:05.914997 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:05.915009 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:05.915069 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:05.915404 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.915923 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.916611 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:05.916922 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:05.916989 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:05.919674 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:45049: BlockingRecv error: recv got EOF from 127.23.186.193:45049 (error 108)
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
Aug 11 14:31:06 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:06 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:06.043563 2624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:06.038444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:51621)
0811 14:31:06.038620 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:06.038631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:06.038690 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:06.038844 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:06.038845 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:06.039043 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:06.039096 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:06.039098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.039923 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.039924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:06.040483 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:06.040485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.040832 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.040833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:06.040888 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:06.041450 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:06.041455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:06.043490 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":110,"threads_started":1}
W20260811 14:31:06.043689 2623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:06.038573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:51621 (local address 127.23.186.254:33769)
0811 14:31:06.038642 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:06.038644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:06.038677 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:31:06.038742 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:06.038743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:06.038755 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:06.038819 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:06.039171 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.039824 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.040562 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.040766 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.040818 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:06.043588 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:51621: BlockingRecv error: recv got EOF from 127.23.186.194:51621 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 14:31:06 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:06 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:06.176100 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:06.170768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:53515)
0811 14:31:06.170997 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:06.171013 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:06.171120 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:06.171308 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:06.171309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:06.171532 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:06.171584 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:06.171586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.172415 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.172417 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:06.172984 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:06.172987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.173373 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.173374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:06.173439 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:06.173923 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:06.173928 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:06.176037 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":123,"threads_started":1}
W20260811 14:31:06.176241 2623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:06.170907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53515 (local address 127.23.186.254:33769)
0811 14:31:06.171001 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:06.171004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:06.171102 (+ 98us) server_negotiation.cc:407] Connection header received
0811 14:31:06.171207 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:06.171208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:06.171230 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:06.171299 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:06.171670 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.172297 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.173097 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.173293 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.173374 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:06.176154 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:53515: BlockingRecv error: recv got EOF from 127.23.186.195:53515 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 14:31:06 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:06 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:06.931540 2627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:06.920159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:55987)
0811 14:31:06.920302 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:06.920322 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:06.920375 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:06.920577 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:06.920578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:06.920785 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:06.920849 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:06.920851 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.921716 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.921717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:06.922289 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:06.922291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.922669 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.922670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:06.922712 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:06.929410 (+ 6698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:06.929417 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:06.931458 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
W20260811 14:31:06.931682 2628 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:06.920258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:55987 (local address 127.23.186.254:33769)
0811 14:31:06.920431 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:06.920433 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:06.920443 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:06.920471 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:06.920472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:06.920483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:06.920568 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:06.920924 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.921605 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.922379 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:06.922592 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:06.922652 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:06.931596 (+ 8944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:55987: BlockingRecv error: recv got EOF from 127.23.186.193:55987 (error 108)
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":106,"threads_started":1}
Aug 11 14:31:07 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:07 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:07.049357 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:07.044101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:43999)
0811 14:31:07.044339 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:07.044358 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:07.044488 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:07.044662 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:07.044664 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:07.044891 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:07.044942 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:07.044944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.045641 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.045642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:07.046175 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:07.046177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.046566 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.046567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:07.046628 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:07.047167 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:07.047172 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:07.049294 (+ 2122us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":113,"threads_started":1}
W20260811 14:31:07.049487 2628 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:07.044230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:43999 (local address 127.23.186.254:33769)
0811 14:31:07.044344 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:07.044347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:07.044464 (+ 117us) server_negotiation.cc:407] Connection header received
0811 14:31:07.044567 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:07.044568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:07.044588 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:07.044655 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:07.045031 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.045552 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.046272 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.046488 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.046567 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:07.049384 (+ 2817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:43999: BlockingRecv error: recv got EOF from 127.23.186.194:43999 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 14:31:07 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:07 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:07.182029 2630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:07.176792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:49639)
0811 14:31:07.176974 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:07.176988 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:07.177096 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:07.177283 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:07.177284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:07.177506 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:07.177557 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:07.177559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.178280 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.178281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:07.178828 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:07.178831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.179203 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.179204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:07.179265 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:07.179751 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:07.179755 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:07.181917 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":79,"threads_started":1}
W20260811 14:31:07.182086 2628 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:07.176832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:49639 (local address 127.23.186.254:33769)
0811 14:31:07.176962 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:07.176964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:07.177075 (+ 111us) server_negotiation.cc:407] Connection header received
0811 14:31:07.177189 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:07.177190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:07.177208 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:07.177276 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:07.177660 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.178183 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.178920 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.179126 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.179204 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:07.182018 (+ 2814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:49639: BlockingRecv error: recv got EOF from 127.23.186.195:49639 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 14:31:07 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:07 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:07.937441 2631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:07.932141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:41095)
0811 14:31:07.932356 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:07.932369 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:07.932442 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:07.932610 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:07.932611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:07.932899 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:07.932969 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:07.932971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.933695 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.933697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:07.934284 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:07.934286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.934672 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.934673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:07.934721 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:07.935242 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:07.935247 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:07.937379 (+ 2132us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":131,"threads_started":1}
W20260811 14:31:07.937640 2632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:07.932231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:41095 (local address 127.23.186.254:33769)
0811 14:31:07.932368 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:07.932371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:07.932425 (+ 54us) server_negotiation.cc:407] Connection header received
0811 14:31:07.932507 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:07.932507 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:07.932520 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:07.932586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:07.933063 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.933606 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.934380 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:07.934593 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:07.934657 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:07.937520 (+ 2863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:41095: BlockingRecv error: recv got EOF from 127.23.186.193:41095 (error 108)
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
Aug 11 14:31:08 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:08 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:08.055099 2633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:08.049891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:54785)
0811 14:31:08.050136 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:08.050150 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:08.050259 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:08.050449 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:08.050451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:08.050656 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:08.050704 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:08.050705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.051406 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.051407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:08.051934 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:08.051936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.052303 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.052304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:08.052362 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:08.052857 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:08.052862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:08.055022 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":126,"threads_started":1}
W20260811 14:31:08.055164 2632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:08.050048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:54785 (local address 127.23.186.254:33769)
0811 14:31:08.050148 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:08.050151 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:08.050237 (+ 86us) server_negotiation.cc:407] Connection header received
0811 14:31:08.050347 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:08.050350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:08.050363 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:08.050441 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:08.050794 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.051316 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.052023 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.052224 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.052305 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:08.055116 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:54785: BlockingRecv error: recv got EOF from 127.23.186.194:54785 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 14:31:08 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:08 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:08.187994 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:08.182660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:53645)
0811 14:31:08.182883 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:08.182896 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:08.182990 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:08.183186 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:08.183187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:08.183363 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:08.183416 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:08.183418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.184140 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.184140 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:08.184666 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:08.184667 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.185106 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.185107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:08.185197 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:08.185781 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:08.185787 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:08.187907 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":125,"threads_started":1}
W20260811 14:31:08.188102 2632 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:08.182839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53645 (local address 127.23.186.254:33769)
0811 14:31:08.182900 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:08.182902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:08.182948 (+ 46us) server_negotiation.cc:407] Connection header received
0811 14:31:08.183057 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:08.183058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:08.183071 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:08.183121 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:08.183526 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.184063 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.184845 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.185037 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.185091 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:08.188022 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:53645: BlockingRecv error: recv got EOF from 127.23.186.195:53645 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 11 14:31:08 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:08 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:08.943307 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:08.938178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:51193)
0811 14:31:08.938350 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:08.938368 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:08.938451 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:08.938645 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:08.938647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:08.938932 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:08.938996 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:08.938998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.939677 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.939678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:08.940229 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:08.940230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.940598 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.940599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:08.940641 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:08.941184 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:08.941189 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:08.943178 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
W20260811 14:31:08.943362 2636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:08.938247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:51193 (local address 127.23.186.254:33769)
0811 14:31:08.938397 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:08.938400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:08.938430 (+ 30us) server_negotiation.cc:407] Connection header received
0811 14:31:08.938532 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:08.938533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:08.938547 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:08.938619 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:08.939073 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.939596 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.940315 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:08.940525 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:08.940584 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:08.943278 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:51193: BlockingRecv error: recv got EOF from 127.23.186.193:51193 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
Aug 11 14:31:09 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:09 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:09.060885 2637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:09.055687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:59499)
0811 14:31:09.055852 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:09.055865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:09.055949 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:09.056105 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:09.056106 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:09.056301 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:09.056355 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:09.056356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.057159 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.057160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:09.057716 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:09.057718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.058059 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.058059 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:09.058101 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:09.058645 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:09.058657 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:09.060810 (+ 2153us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
W20260811 14:31:09.061020 2636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:09.055813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:59499 (local address 127.23.186.254:33769)
0811 14:31:09.055897 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:09.055899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:09.055932 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:31:09.056012 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:09.056013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:09.056024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:09.056079 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:09.056426 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.057057 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.057800 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.057989 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.058041 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:09.060917 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:59499: BlockingRecv error: recv got EOF from 127.23.186.194:59499 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 14:31:09 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:09 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:09.193883 2639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:09.188626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:53055)
0811 14:31:09.188851 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:09.188862 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:09.188948 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:09.189109 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:09.189109 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:09.189288 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:09.189351 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:09.189353 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.190138 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.190139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:09.190693 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:09.190694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.191079 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.191080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:09.191125 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:09.191697 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:09.191702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:09.193810 (+ 2108us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":157,"threads_started":1}
W20260811 14:31:09.194018 2636 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:09.188740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53055 (local address 127.23.186.254:33769)
0811 14:31:09.188812 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:09.188815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:09.188978 (+ 163us) server_negotiation.cc:407] Connection header received
0811 14:31:09.189020 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:09.189021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:09.189038 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:09.189096 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:09.189449 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.190058 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.190770 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.191011 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.191070 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:09.193913 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:53055: BlockingRecv error: recv got EOF from 127.23.186.195:53055 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 11 14:31:09 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:09 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:09.949205 2640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:09.943986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:54731)
0811 14:31:09.944188 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:09.944200 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:09.944272 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:09.944459 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:09.944460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:09.944646 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:09.944728 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:09.944730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.945411 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.945412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:09.945947 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:09.945949 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.946546 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.946547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:09.946619 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:09.947213 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:09.947219 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:09.949142 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":116,"threads_started":1}
W20260811 14:31:09.949349 2641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:09.944153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:54731 (local address 127.23.186.254:33769)
0811 14:31:09.944306 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:09.944308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:09.944318 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:09.944356 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:09.944356 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:09.944368 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:09.944444 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:09.944810 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.945333 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.946038 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:09.946347 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:09.946418 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:09.949256 (+ 2838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:54731: BlockingRecv error: recv got EOF from 127.23.186.193:54731 (error 108)
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":84,"threads_started":1}
Aug 11 14:31:10 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:10 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:10.066591 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:10.061426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:55969)
0811 14:31:10.061616 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:10.061633 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:10.061689 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:10.061866 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:10.061867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:10.062258 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:10.062314 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:10.062317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.063020 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.063021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:10.063592 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:10.063593 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.063926 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.063927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:10.063968 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:10.064556 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:10.064562 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:10.066513 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":125,"threads_started":1}
W20260811 14:31:10.066776 2641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:10.061563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:55969 (local address 127.23.186.254:33769)
0811 14:31:10.061736 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:10.061738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:10.061747 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:31:10.061774 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:10.061775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:10.061787 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:10.061847 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:10.062421 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.062939 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.063672 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.063859 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.063911 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:10.066696 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:55969: BlockingRecv error: recv got EOF from 127.23.186.194:55969 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 14:31:10 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:10 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:10.199146 2645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:10.194389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:35589)
0811 14:31:10.194521 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:10.194535 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:10.194611 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:10.194774 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:10.194775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:10.194958 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:10.195010 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:10.195012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.195689 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.195690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:10.196234 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:10.196236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.196579 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.196579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:10.196621 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:10.197253 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:10.197258 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:10.199083 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":61,"threads_started":1}
W20260811 14:31:10.199275 2641 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:10.194492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:35589 (local address 127.23.186.254:33769)
0811 14:31:10.194597 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:10.194600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:10.194617 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:31:10.194682 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:10.194683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:10.194697 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:10.194747 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:10.195076 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.195599 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.196313 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.196500 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.196559 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:10.199192 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:35589: BlockingRecv error: recv got EOF from 127.23.186.195:35589 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 14:31:10 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:10 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:10.954694 2646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:10.949833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:58961)
0811 14:31:10.949988 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:10.950000 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:10.950066 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:10.950260 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:10.950261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:10.950524 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:10.950612 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:10.950615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.951398 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.951399 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:10.951934 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:10.951935 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.952282 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.952283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:10.952325 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:10.952819 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:10.952829 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:10.954631 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
W20260811 14:31:10.954813 2647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:10.949931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:58961 (local address 127.23.186.254:33769)
0811 14:31:10.950073 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:10.950075 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:10.950116 (+ 41us) server_negotiation.cc:407] Connection header received
0811 14:31:10.950143 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:10.950143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:10.950154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:10.950224 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:10.950721 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.951318 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.952018 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:10.952213 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:10.952266 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:10.954733 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:58961: BlockingRecv error: recv got EOF from 127.23.186.193:58961 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":88,"threads_started":1}
Aug 11 14:31:11 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:11 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:11.072440 2648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:11.067264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:38513)
0811 14:31:11.067515 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:11.067530 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:11.067622 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:11.067774 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:11.067775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:11.067990 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:11.068042 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:11.068045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.068814 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.068815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:11.069358 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:11.069360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.069780 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.069781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:11.069843 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:11.070420 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:11.070427 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:11.072378 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":87,"threads_started":1}
W20260811 14:31:11.072674 2647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:11.067390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:38513 (local address 127.23.186.254:33769)
0811 14:31:11.067444 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:11.067447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:11.067604 (+ 157us) server_negotiation.cc:407] Connection header received
0811 14:31:11.067699 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:11.067700 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:11.067713 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:11.067771 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:11.068139 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.068661 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.069475 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.069682 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.069779 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:11.072613 (+ 2834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:38513: BlockingRecv error: recv got EOF from 127.23.186.194:38513 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 11 14:31:11 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:11 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:11.204317 2649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:11.199760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:46893)
0811 14:31:11.199905 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:11.199917 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:11.199983 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:11.200104 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:11.200105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:11.200283 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:11.200336 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:11.200338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.200991 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.200991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:11.201534 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:11.201536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.201856 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.201857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:11.201899 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:11.202351 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:11.202355 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:11.204249 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
W20260811 14:31:11.204423 2647 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:11.199825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:46893 (local address 127.23.186.254:33769)
0811 14:31:11.199883 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:11.199885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:11.199984 (+ 99us) server_negotiation.cc:407] Connection header received
0811 14:31:11.200038 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:11.200039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:11.200052 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:11.200094 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:11.200394 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.200922 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.201611 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.201792 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.201842 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:11.204352 (+ 2510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:46893: BlockingRecv error: recv got EOF from 127.23.186.195:46893 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 11 14:31:11 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:11 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:11.965727 2650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:11.955376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:55649)
0811 14:31:11.955525 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:11.955536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:11.955595 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:11.955758 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:11.955760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:11.955941 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:11.956010 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:11.956013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.956737 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.956738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:11.957343 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:11.957346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.957682 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.957683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:11.957724 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:11.963507 (+ 5783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:11.963514 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:11.965616 (+ 2102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260811 14:31:11.965806 2651 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:11.955432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:55649 (local address 127.23.186.254:33769)
0811 14:31:11.955586 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:11.955589 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:11.955599 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:11.955660 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:11.955661 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:11.955672 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:11.955738 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:11.956077 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.956620 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.957414 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:11.957613 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:11.957669 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:11.965729 (+ 8060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:55649: BlockingRecv error: recv got EOF from 127.23.186.193:55649 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":91,"threads_started":1}
Aug 11 14:31:12 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:12 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:12.077728 2653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:12.072993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:54865)
0811 14:31:12.073217 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:12.073232 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:12.073301 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:12.073471 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:12.073472 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:12.073651 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:12.073702 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:12.073704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.074401 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.074402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:12.074962 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:12.074964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.075293 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.075294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:12.075334 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:12.075790 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:12.075794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:12.077657 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":85,"threads_started":1}
W20260811 14:31:12.077834 2651 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:12.073239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:54865 (local address 127.23.186.254:33769)
0811 14:31:12.073328 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:12.073330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:12.073340 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:12.073378 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:12.073378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:12.073389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:12.073444 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:12.073798 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.074318 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.075042 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.075227 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.075278 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:12.077764 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:54865: BlockingRecv error: recv got EOF from 127.23.186.194:54865 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 14:31:12 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:12 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.195@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:12.209873 2654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:12.205044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.195:32837)
0811 14:31:12.205220 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:12.205238 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:12.205304 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:12.205449 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:12.205450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:12.205639 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:12.205693 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:12.205694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.206359 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.206359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:12.206907 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:12.206909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.207264 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.207265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:12.207307 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:12.207794 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:12.207799 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:12.209736 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
W20260811 14:31:12.209899 2651 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:12.205131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:32837 (local address 127.23.186.254:33769)
0811 14:31:12.205209 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:12.205211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:12.205303 (+ 92us) server_negotiation.cc:407] Connection header received
0811 14:31:12.205354 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:12.205355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:12.205369 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:12.205426 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:12.205763 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.206283 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.206995 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.207190 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.207250 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:12.209829 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.195:32837: BlockingRecv error: recv got EOF from 127.23.186.195:32837 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 14:31:12 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:12 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.193@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:12.971517 2656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:12.966384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.193:59031)
0811 14:31:12.966548 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:12.966562 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:12.966623 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:12.966777 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:12.966778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:12.966978 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:12.967042 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:12.967044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.967751 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.967752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:12.968314 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:12.968316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.968673 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.968674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:12.968783 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:12.969420 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:12.969425 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:12.971447 (+ 2022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":127,"threads_started":1}
W20260811 14:31:12.971670 2657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:12.966489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:59031 (local address 127.23.186.254:33769)
0811 14:31:12.966630 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:12.966631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:12.966643 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:31:12.966671 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:12.966671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:12.966682 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:12.966752 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:12.967123 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.967658 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.968396 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:12.968600 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:12.968656 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:12.971576 (+ 2920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.193:59031: BlockingRecv error: recv got EOF from 127.23.186.193:59031 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":83,"threads_started":1}
Aug 11 14:31:13 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 14:31:13 dist-test-slave-cbsf krb5kdc[1392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.23.186.194@KRBTEST.COM for oryx/127.23.186.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 14:31:13.083983 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.078561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33769 (local address 127.23.186.194:49133)
0811 14:31:13.078708 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:13.078751 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:13.078821 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:13.079047 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:13.079048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:13.079317 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:13.079389 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.079392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.080112 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.080113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.080655 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.080656 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.081094 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.081095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.081140 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.081714 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:13.081723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:13.083919 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.23.186.254:33769: Server oryx/127.23.186.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
W20260811 14:31:13.084219 2657 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:13.078723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:49133 (local address 127.23.186.254:33769)
0811 14:31:13.078791 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.078793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.078805 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:31:13.078904 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.078905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.078920 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.079010 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.079487 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.080006 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.080787 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.081021 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.081081 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.084131 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.23.186.194:49133: BlockingRecv error: recv got EOF from 127.23.186.194:49133 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:31:13.122213 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2096
I20260811 14:31:13.129807 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2274
I20260811 14:31:13.137758 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2435
I20260811 14:31:13.144398 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2008
2026-08-11T14:31:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17678 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T14:31:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:31:13Z Disabled control of system clock
I20260811 14:31:13.178397 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41333
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:35163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:41333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 14:31:13.264336 2664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:13.264583 2664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:13.264605 2664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:13.265904 2664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:31:13.265959 2664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:13.265972 2664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:13.265985 2664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:31:13.265997 2664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:31:13.267462 2664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:41333
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:13.267774 2664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:13.268117 2664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:13.273456 2670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.273530 2664 server_base.cc:1034] running on GCE node
W20260811 14:31:13.273535 2669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.273456 2672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.274034 2664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:13.274381 2664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:13.275538 2664 hybrid_clock.cc:648] HybridClock initialized: now 1786458673275522 us; error 38 us; skew 500 ppm
I20260811 14:31:13.276940 2664 webserver.cc:460] Webserver started at http://127.23.186.254:33583/ using document root <none> and password file <none>
I20260811 14:31:13.277273 2664 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:13.277348 2664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:13.277516 2664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:13.278662 2664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "6b8334122151435489672bf43cc80a7f"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "0a177c27233f47b32cdd27e7acd8400c"
server_key_iv: "aac97ef6945f88e8a100d9b8678cee1c"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.279052 2664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "6b8334122151435489672bf43cc80a7f"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "0a177c27233f47b32cdd27e7acd8400c"
server_key_iv: "aac97ef6945f88e8a100d9b8678cee1c"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.280452 2664 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:13.281350 2679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.281539 2664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:31:13.281651 2664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "6b8334122151435489672bf43cc80a7f"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "0a177c27233f47b32cdd27e7acd8400c"
server_key_iv: "aac97ef6945f88e8a100d9b8678cee1c"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.281744 2664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:13.298301 2664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:13.298671 2664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:13.298843 2664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:13.303229 2664 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:41333
I20260811 14:31:13.303267 2740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:41333 every 8 connection(s)
I20260811 14:31:13.303668 2664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:31:13.304388 2741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:31:13.307165 2741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f: Bootstrap starting.
I20260811 14:31:13.307739 2741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f: Neither blocks nor log segments found. Creating new log.
I20260811 14:31:13.307976 2741 log.cc:824] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:31:13.308682 2741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f: No bootstrap required, opened a new log
I20260811 14:31:13.310424 2741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8334122151435489672bf43cc80a7f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41333 } }
I20260811 14:31:13.310580 2741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:31:13.310603 2741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b8334122151435489672bf43cc80a7f, State: Initialized, Role: FOLLOWER
I20260811 14:31:13.310801 2741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8334122151435489672bf43cc80a7f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41333 } }
I20260811 14:31:13.310885 2741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:31:13.310933 2741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:31:13.310982 2741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:31:13.311578 2741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8334122151435489672bf43cc80a7f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41333 } }
I20260811 14:31:13.311694 2741 leader_election.cc:302] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b8334122151435489672bf43cc80a7f; no voters:
I20260811 14:31:13.311879 2741 leader_election.cc:288] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:31:13.312165 2745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:31:13.312347 2745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [term 1 LEADER]: Becoming Leader. State: Replica: 6b8334122151435489672bf43cc80a7f, State: Running, Role: LEADER
I20260811 14:31:13.312387 2741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:31:13.312497 2745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8334122151435489672bf43cc80a7f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41333 } }
I20260811 14:31:13.313014 2747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b8334122151435489672bf43cc80a7f. Latest consensus state: current_term: 1 leader_uuid: "6b8334122151435489672bf43cc80a7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8334122151435489672bf43cc80a7f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41333 } } }
I20260811 14:31:13.313115 2747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [sys.catalog]: This master's current role is: LEADER
I20260811 14:31:13.313277 2746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b8334122151435489672bf43cc80a7f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b8334122151435489672bf43cc80a7f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41333 } } }
I20260811 14:31:13.313331 2756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:31:13.313864 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2664
I20260811 14:31:13.313963 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:31:13.314174 24299 external_mini_cluster.cc:1428] Setting key 203d560d09156d9906f70dcd86f26a26
I20260811 14:31:13.313637 2746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f [sys.catalog]: This master's current role is: LEADER
I20260811 14:31:13.314237 2756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:31:13.315907 2756 catalog_manager.cc:1347] Generated new cluster ID: ce53f269fa984b41b8b282b782e45567
I20260811 14:31:13.315964 2756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:31:13.324887 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.314872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.23.186.254:41333)
0811 14:31:13.315090 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.315099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.315132 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:31:13.315248 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.315256 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.315327 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.315480 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.315762 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.321764 (+ 6002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.322638 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.323099 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.323246 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.324034 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:13.324039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:13.324039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:13.324233 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:13.324270 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:13.324283 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:13.324362 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:13.324442 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:31:13.324637 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260811 14:31:13.345988 2756 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:31:13.346648 2756 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:31:13.349821 2756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b8334122151435489672bf43cc80a7f: Generated new TSK 0
I20260811 14:31:13.350003 2756 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:31:13.356029 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41333
--builtin_ntp_servers=127.23.186.212:35163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 14:31:13.441803 2772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:13.442051 2772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:13.442092 2772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:13.443323 2772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:13.443379 2772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:13.443424 2772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:31:13.444911 2772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:13.445178 2772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:13.445495 2772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:13.451426 2780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.451455 2784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.451463 2782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.451826 2772 server_base.cc:1034] running on GCE node
I20260811 14:31:13.452041 2772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:13.452304 2772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:13.453522 2772 hybrid_clock.cc:648] HybridClock initialized: now 1786458673453515 us; error 58 us; skew 500 ppm
I20260811 14:31:13.455080 2772 webserver.cc:460] Webserver started at http://127.23.186.193:44355/ using document root <none> and password file <none>
I20260811 14:31:13.455372 2772 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:13.455446 2772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:13.455618 2772 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:13.456827 2772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "a00f05f8eb0f4d64b27b64f06b968a98"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "57a5302312567d3a7c3122fa7e1c63ef"
server_key_iv: "3cc0e15c652678cb0c752d091c29f901"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.457211 2772 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "a00f05f8eb0f4d64b27b64f06b968a98"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "57a5302312567d3a7c3122fa7e1c63ef"
server_key_iv: "3cc0e15c652678cb0c752d091c29f901"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.458613 2772 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:31:13.459373 2791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.459548 2772 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:31:13.459658 2772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "a00f05f8eb0f4d64b27b64f06b968a98"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "57a5302312567d3a7c3122fa7e1c63ef"
server_key_iv: "3cc0e15c652678cb0c752d091c29f901"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.459739 2772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:13.468107 2772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:13.468454 2772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:13.468626 2772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:13.468884 2772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:13.469256 2772 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:13.469312 2772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.469357 2772 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:13.469388 2772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.475793 2772 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:36143
I20260811 14:31:13.475839 2923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:36143 every 8 connection(s)
I20260811 14:31:13.476238 2772 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:31:13.481967 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.477313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:53897 (local address 127.23.186.254:41333)
0811 14:31:13.477467 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.477469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.477696 (+ 227us) server_negotiation.cc:407] Connection header received
0811 14:31:13.478191 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.478192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.478206 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.478273 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.478951 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.479459 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.480476 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.480671 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.480812 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.481740 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:13.481741 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:13.481742 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:13.481746 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:13.481757 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:13.481761 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:13.481800 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:13.481865 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:31:13.481902 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 14:31:13.482208 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.477151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41333 (local address 127.23.186.193:53897)
0811 14:31:13.477574 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:13.477613 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:13.478104 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:13.478326 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:13.478329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:13.478338 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:13.478868 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.478873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.479570 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.479571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.480387 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.480389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.480835 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.480836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.480959 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.481595 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:31:13.481596 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:13.481631 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:31:13.481657 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:13.481802 (+ 145us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:13.481804 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:13.481807 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:31:13.481855 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:31:13.481943 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":108,"threads_started":1}
I20260811 14:31:13.482316 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2772
I20260811 14:31:13.482407 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:31:13.482616 24299 external_mini_cluster.cc:1428] Setting key 7d8f1a09387c5710561b08d0543649c5
I20260811 14:31:13.482743 2924 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41333
I20260811 14:31:13.482858 2924 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:13.483094 2924 heartbeater.cc:511] Master 127.23.186.254:41333 requested a full tablet report, sending...
I20260811 14:31:13.483656 2699 ts_manager.cc:194] Registered new tserver with Master: a00f05f8eb0f4d64b27b64f06b968a98 (127.23.186.193:36143)
I20260811 14:31:13.483687 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41333
--builtin_ntp_servers=127.23.186.212:35163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 14:31:13.484360 2699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.193:53897
W20260811 14:31:13.568666 2927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:13.568951 2927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:13.568974 2927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:13.570225 2927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:13.570272 2927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:13.570300 2927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:31:13.571691 2927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:13.571959 2927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:13.572278 2927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:13.577592 2932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.577585 2933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.577591 2935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.577613 2927 server_base.cc:1034] running on GCE node
I20260811 14:31:13.578133 2927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:13.578426 2927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:13.579598 2927 hybrid_clock.cc:648] HybridClock initialized: now 1786458673579592 us; error 40 us; skew 500 ppm
I20260811 14:31:13.581067 2927 webserver.cc:460] Webserver started at http://127.23.186.194:43301/ using document root <none> and password file <none>
I20260811 14:31:13.581334 2927 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:13.581409 2927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:13.581573 2927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:13.582718 2927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "3550356afb4c4ae0979f01b90e5e44fc"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "cc3107a0432ed65aa8ca86b5f4749c16"
server_key_iv: "34725dbe7a2826518b7107b87f5634c3"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.583096 2927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "3550356afb4c4ae0979f01b90e5e44fc"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "cc3107a0432ed65aa8ca86b5f4749c16"
server_key_iv: "34725dbe7a2826518b7107b87f5634c3"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.584494 2927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:13.585387 2941 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.585558 2927 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:31:13.585666 2927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "3550356afb4c4ae0979f01b90e5e44fc"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "cc3107a0432ed65aa8ca86b5f4749c16"
server_key_iv: "34725dbe7a2826518b7107b87f5634c3"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.585758 2927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:13.599071 2927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:13.599454 2927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:13.599627 2927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:13.599881 2927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:13.600238 2927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:13.600296 2927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.600342 2927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:13.600364 2927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.606632 2927 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:43673
I20260811 14:31:13.606673 3079 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:43673 every 8 connection(s)
I20260811 14:31:13.607064 2927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:31:13.609840 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 2927
I20260811 14:31:13.609954 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:31:13.610172 24299 external_mini_cluster.cc:1428] Setting key e61b2d8a6904fc7082e0ac9fde5eb63c
I20260811 14:31:13.611279 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41333
--builtin_ntp_servers=127.23.186.212:35163
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 14:31:13.614084 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.607851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:55549 (local address 127.23.186.254:41333)
0811 14:31:13.607943 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.607945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.608184 (+ 239us) server_negotiation.cc:407] Connection header received
0811 14:31:13.609077 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.609078 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.609089 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.609134 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.610068 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.610815 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.612398 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.612588 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.612639 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.613760 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:13.613762 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:13.613762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:13.613769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:13.613788 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:13.613793 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:13.613855 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:13.613941 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 14:31:13.613975 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 14:31:13.614288 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.607808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41333 (local address 127.23.186.194:55549)
0811 14:31:13.608028 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:13.608064 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:13.608913 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:13.609214 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:13.609219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:13.609230 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:13.609953 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.609960 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.610935 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.610936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.612303 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.612307 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.612734 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.612736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.612906 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.613599 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:31:13.613599 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:13.613636 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:31:13.613662 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:13.613861 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:13.613865 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:13.613868 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:31:13.613922 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:31:13.614010 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
I20260811 14:31:13.614822 3080 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41333
I20260811 14:31:13.614924 3080 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:13.615137 3080 heartbeater.cc:511] Master 127.23.186.254:41333 requested a full tablet report, sending...
I20260811 14:31:13.615547 2699 ts_manager.cc:194] Registered new tserver with Master: 3550356afb4c4ae0979f01b90e5e44fc (127.23.186.194:43673)
I20260811 14:31:13.615933 2699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.194:55549
W20260811 14:31:13.698958 3083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:13.699213 3083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:13.699247 3083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:13.700419 3083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:13.700486 3083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:13.700529 3083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:31:13.701923 3083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35163
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41333
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:13.702204 3083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:13.702520 3083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:13.707808 3090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.707846 3089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.707839 3083 server_base.cc:1034] running on GCE node
W20260811 14:31:13.707818 3092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.708273 3083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:13.708531 3083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:13.709682 3083 hybrid_clock.cc:648] HybridClock initialized: now 1786458673709671 us; error 43 us; skew 500 ppm
I20260811 14:31:13.711059 3083 webserver.cc:460] Webserver started at http://127.23.186.195:44855/ using document root <none> and password file <none>
I20260811 14:31:13.711311 3083 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:13.711360 3083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:13.711537 3083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:13.712852 3083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "e1ee5017b484430da8e2d2bf3287dcd5"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "1dc3313b3deae1b721dec908d69c038b"
server_key_iv: "4802f459df2ba4b7749a82d90693f752"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.713253 3083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "e1ee5017b484430da8e2d2bf3287dcd5"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "1dc3313b3deae1b721dec908d69c038b"
server_key_iv: "4802f459df2ba4b7749a82d90693f752"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.714654 3083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:31:13.715374 3098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.715549 3083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:31:13.715647 3083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "e1ee5017b484430da8e2d2bf3287dcd5"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "1dc3313b3deae1b721dec908d69c038b"
server_key_iv: "4802f459df2ba4b7749a82d90693f752"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.715724 3083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:13.733139 3083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:13.733485 3083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:13.733656 3083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:13.733902 3083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:13.734261 3083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:13.734297 3083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.734320 3083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:13.734369 3083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.740312 3083 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:44173
I20260811 14:31:13.740366 3230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:44173 every 8 connection(s)
I20260811 14:31:13.740752 3083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:31:13.745832 2770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.741501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:43775 (local address 127.23.186.254:41333)
0811 14:31:13.741555 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.741557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.741834 (+ 277us) server_negotiation.cc:407] Connection header received
0811 14:31:13.742308 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.742309 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.742319 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.742382 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.743071 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.743581 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.744545 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.744752 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.744801 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.745570 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:13.745571 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:13.745571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:13.745597 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:13.745611 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:13.745615 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:13.745657 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:13.745742 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 14:31:13.745769 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:31:13.746079 3233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.741476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41333 (local address 127.23.186.195:43775)
0811 14:31:13.741695 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:13.741738 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:13.742226 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:13.742451 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:13.742454 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:13.742463 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:13.742993 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.742997 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.743666 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.743666 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.744465 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:13.744467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:13.744812 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:13.744813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:13.744927 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:13.745432 (+ 505us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:31:13.745432 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:13.745461 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:31:13.745484 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:13.745671 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:13.745675 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:13.745678 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:31:13.745731 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:31:13.745817 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
I20260811 14:31:13.746659 3231 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41333
I20260811 14:31:13.746768 3231 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:13.746958 3231 heartbeater.cc:511] Master 127.23.186.254:41333 requested a full tablet report, sending...
I20260811 14:31:13.747382 2699 ts_manager.cc:194] Registered new tserver with Master: e1ee5017b484430da8e2d2bf3287dcd5 (127.23.186.195:44173)
I20260811 14:31:13.747779 2699 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.195:43775
I20260811 14:31:13.748442 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3083
I20260811 14:31:13.748540 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:31:13.748793 24299 external_mini_cluster.cc:1428] Setting key 37e91b1117c0cb9d0bf4e322fcb629a1
I20260811 14:31:13.749539 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 14:31:13.751545 2770 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:13.750976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54294 (local address 127.23.186.254:41333)
0811 14:31:13.751044 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.751046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.751188 (+ 142us) server_negotiation.cc:407] Connection header received
0811 14:31:13.751231 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.751232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.751244 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.751299 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.751458 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54294: BlockingRecv error: recv got EOF from 127.0.0.1:54294 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:31:13.751570 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2772
I20260811 14:31:13.760578 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2927
I20260811 14:31:13.769608 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3083
I20260811 14:31:13.779111 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 2664
2026-08-11T14:31:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (633 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T14:31:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:31:13Z Disabled control of system clock
I20260811 14:31:13.811542 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:32929
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:32929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 14:31:13.897642 3246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:13.897882 3246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:13.897915 3246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:13.899089 3246 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:31:13.899130 3246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:13.899143 3246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:13.899156 3246 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:31:13.899169 3246 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:31:13.900542 3246 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:32929
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:32929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:13.900895 3246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:13.901183 3246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:13.906361 3252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:13.906356 3254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.906365 3246 server_base.cc:1034] running on GCE node
W20260811 14:31:13.906481 3251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:13.906813 3246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:13.907090 3246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:13.908277 3246 hybrid_clock.cc:648] HybridClock initialized: now 1786458673908264 us; error 50 us; skew 500 ppm
I20260811 14:31:13.909749 3246 webserver.cc:460] Webserver started at http://127.23.186.254:41125/ using document root <none> and password file <none>
I20260811 14:31:13.910029 3246 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:13.910112 3246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:13.910267 3246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:13.911373 3246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "2a6f92d175824d81892347c5b06cd169"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "ea76a9e92dae16311291af8c73d3b6a8"
server_key_iv: "9bcd44566112875e7a281db6f7e34af0"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.911751 3246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "2a6f92d175824d81892347c5b06cd169"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "ea76a9e92dae16311291af8c73d3b6a8"
server_key_iv: "9bcd44566112875e7a281db6f7e34af0"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.913218 3246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:31:13.913978 3261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:13.914146 3246 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:31:13.914256 3246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "2a6f92d175824d81892347c5b06cd169"
format_stamp: "Formatted at 2026-08-11 14:31:13 on dist-test-slave-cbsf"
server_key: "ea76a9e92dae16311291af8c73d3b6a8"
server_key_iv: "9bcd44566112875e7a281db6f7e34af0"
server_key_version: "encryptionkey@0"
I20260811 14:31:13.914337 3246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:13.918080 3246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:13.918385 3246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:13.918538 3246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:13.923122 3246 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:32929
I20260811 14:31:13.923189 3325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:32929 every 8 connection(s)
I20260811 14:31:13.923571 3246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:31:13.924285 3326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:31:13.926626 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3246
I20260811 14:31:13.926740 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:31:13.927016 24299 external_mini_cluster.cc:1428] Setting key c05c83c307843c1b38bb85a659f99c82
I20260811 14:31:13.927513 3326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169: Bootstrap starting.
I20260811 14:31:13.928248 3326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169: Neither blocks nor log segments found. Creating new log.
I20260811 14:31:13.928612 3326 log.cc:824] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169: Log is configured to *not* fsync() on all Append() calls
I20260811 14:31:13.929461 3326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169: No bootstrap required, opened a new log
I20260811 14:31:13.929998 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:13.927657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.23.186.254:32929)
0811 14:31:13.927879 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:13.927886 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:13.927920 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:31:13.928401 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:13.928412 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:13.928468 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:13.928592 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:13.929380 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:13.929384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:13.929384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:13.929389 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:13.929439 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:13.929454 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:13.929556 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:13.929739 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 14:31:13.929807 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260811 14:31:13.931682 3326 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6f92d175824d81892347c5b06cd169" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 32929 } }
I20260811 14:31:13.931824 3326 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:31:13.931847 3326 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6f92d175824d81892347c5b06cd169, State: Initialized, Role: FOLLOWER
I20260811 14:31:13.931983 3326 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6f92d175824d81892347c5b06cd169" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 32929 } }
I20260811 14:31:13.932060 3326 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:31:13.932098 3326 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:31:13.932150 3326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:31:13.932811 3326 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6f92d175824d81892347c5b06cd169" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 32929 } }
I20260811 14:31:13.932931 3326 leader_election.cc:302] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a6f92d175824d81892347c5b06cd169; no voters:
I20260811 14:31:13.933128 3326 leader_election.cc:288] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:31:13.933261 3331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:31:13.933411 3326 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:31:13.933441 3331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [term 1 LEADER]: Becoming Leader. State: Replica: 2a6f92d175824d81892347c5b06cd169, State: Running, Role: LEADER
I20260811 14:31:13.933542 3331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6f92d175824d81892347c5b06cd169" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 32929 } }
I20260811 14:31:13.934033 3333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2a6f92d175824d81892347c5b06cd169. Latest consensus state: current_term: 1 leader_uuid: "2a6f92d175824d81892347c5b06cd169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6f92d175824d81892347c5b06cd169" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 32929 } } }
I20260811 14:31:13.934150 3333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [sys.catalog]: This master's current role is: LEADER
I20260811 14:31:13.934329 3332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2a6f92d175824d81892347c5b06cd169" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a6f92d175824d81892347c5b06cd169" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 32929 } } }
I20260811 14:31:13.934393 3332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169 [sys.catalog]: This master's current role is: LEADER
I20260811 14:31:13.934412 3340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:31:13.935014 3340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:31:13.936601 3340 catalog_manager.cc:1347] Generated new cluster ID: e09a5358cec2426fb0d48e90198aa8df
I20260811 14:31:13.936691 3340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:31:13.943064 3340 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:31:13.943291 3340 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:31:13.949079 3340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2a6f92d175824d81892347c5b06cd169: Generated new TSK 0
I20260811 14:31:13.949347 3340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:31:13.956640 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:32929
--builtin_ntp_servers=127.23.186.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 14:31:14.053725 3354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:14.053952 3354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:14.053979 3354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:14.055240 3354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:14.055289 3354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:14.055317 3354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:31:14.056936 3354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:14.057240 3354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:14.057592 3354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:14.064512 3360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:14.064643 3363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:14.064764 3354 server_base.cc:1034] running on GCE node
W20260811 14:31:14.064885 3359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:14.065146 3354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:14.065444 3354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:14.066625 3354 hybrid_clock.cc:648] HybridClock initialized: now 1786458674066603 us; error 41 us; skew 500 ppm
I20260811 14:31:14.067991 3354 webserver.cc:460] Webserver started at http://127.23.186.193:33083/ using document root <none> and password file <none>
I20260811 14:31:14.068272 3354 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:14.068351 3354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:14.068514 3354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:14.069816 3354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "2ed921086241457ca97a1c2357fb18e7"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "a96f8ab7c63a7aa3630d86c172da0346"
server_key_iv: "5a0527eccc7bd6665ae92de186e33a69"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.070217 3354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "2ed921086241457ca97a1c2357fb18e7"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "a96f8ab7c63a7aa3630d86c172da0346"
server_key_iv: "5a0527eccc7bd6665ae92de186e33a69"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.071725 3354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:14.072649 3370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.072860 3354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:31:14.072968 3354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "2ed921086241457ca97a1c2357fb18e7"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "a96f8ab7c63a7aa3630d86c172da0346"
server_key_iv: "5a0527eccc7bd6665ae92de186e33a69"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.073064 3354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:14.076660 3354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:14.077013 3354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:14.077181 3354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:14.077421 3354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:14.077772 3354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:14.077831 3354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.077872 3354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:14.077898 3354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.084409 3354 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:42487
I20260811 14:31:14.084479 3506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:42487 every 8 connection(s)
I20260811 14:31:14.084901 3354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:31:14.087976 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:14.085671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:36411 (local address 127.23.186.254:32929)
0811 14:31:14.085765 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:14.085768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:14.085983 (+ 215us) server_negotiation.cc:407] Connection header received
0811 14:31:14.086458 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:14.086459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:14.086464 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:14.086518 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:14.087783 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:14.087783 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:14.087784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:14.087787 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:14.087798 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:14.087801 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:14.087843 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:14.087907 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 14:31:14.087921 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 14:31:14.088218 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:14.085634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:32929 (local address 127.23.186.193:36411)
0811 14:31:14.085854 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:14.085891 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:14.086372 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:14.086573 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:14.086576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:14.086585 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:14.087632 (+ 1047us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:31:14.087633 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:14.087670 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:31:14.087702 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:14.087846 (+ 144us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:14.087848 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:14.087851 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:31:14.087904 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:31:14.087954 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":114,"threads_started":1}
I20260811 14:31:14.088831 3507 heartbeater.cc:347] Connected to a master server at 127.23.186.254:32929
I20260811 14:31:14.088944 3507 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:14.089115 3507 heartbeater.cc:511] Master 127.23.186.254:32929 requested a full tablet report, sending...
I20260811 14:31:14.089658 3280 ts_manager.cc:194] Registered new tserver with Master: 2ed921086241457ca97a1c2357fb18e7 (127.23.186.193:42487)
I20260811 14:31:14.094662 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3354
I20260811 14:31:14.094789 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:31:14.094978 24299 external_mini_cluster.cc:1428] Setting key 8345a09dec1050894927aceb58f0296c
I20260811 14:31:14.096036 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:32929
--builtin_ntp_servers=127.23.186.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 14:31:14.185647 3511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:14.185912 3511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:14.185954 3511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:14.187187 3511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:14.187265 3511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:14.187315 3511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:31:14.188815 3511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:14.189105 3511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:14.189433 3511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:14.194715 3518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:14.194716 3519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:14.194990 3511 server_base.cc:1034] running on GCE node
W20260811 14:31:14.194739 3521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:14.195305 3511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:14.195520 3511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:14.196662 3511 hybrid_clock.cc:648] HybridClock initialized: now 1786458674196628 us; error 58 us; skew 500 ppm
I20260811 14:31:14.198050 3511 webserver.cc:460] Webserver started at http://127.23.186.194:38829/ using document root <none> and password file <none>
I20260811 14:31:14.198275 3511 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:14.198318 3511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:14.198413 3511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:14.199487 3511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "03ba8e78aa9f4adb9c56de4a8b3508e5"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "49c7974e8e516af88daad73161a290c7"
server_key_iv: "b2c7bd023ef7b4de52e094c27904d30f"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.199802 3511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "03ba8e78aa9f4adb9c56de4a8b3508e5"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "49c7974e8e516af88daad73161a290c7"
server_key_iv: "b2c7bd023ef7b4de52e094c27904d30f"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.201162 3511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:31:14.201923 3528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.202186 3511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:31:14.202301 3511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "03ba8e78aa9f4adb9c56de4a8b3508e5"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "49c7974e8e516af88daad73161a290c7"
server_key_iv: "b2c7bd023ef7b4de52e094c27904d30f"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.202378 3511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:14.205783 3511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:14.206064 3511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:14.206208 3511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:14.206434 3511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:14.206769 3511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:14.206826 3511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.206867 3511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:14.206895 3511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.212539 3511 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:43935
I20260811 14:31:14.212622 3659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:43935 every 8 connection(s)
I20260811 14:31:14.212998 3511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:31:14.216059 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:14.213791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:53081 (local address 127.23.186.254:32929)
0811 14:31:14.213842 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:14.213844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:14.214098 (+ 254us) server_negotiation.cc:407] Connection header received
0811 14:31:14.214560 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:14.214561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:14.214567 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:14.214621 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:14.215861 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:14.215862 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:14.215863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:14.215867 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:14.215880 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:14.215883 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:14.215927 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:14.215989 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:31:14.216001 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:31:14.216286 3665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:14.213773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:32929 (local address 127.23.186.194:53081)
0811 14:31:14.213960 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:14.213995 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:14.214473 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:14.214674 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:14.214677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:14.214685 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:14.215707 (+ 1022us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:31:14.215707 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:14.215738 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:31:14.215775 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:14.215933 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:14.215935 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:14.215937 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:31:14.215992 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:31:14.216047 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":97,"threads_started":1}
I20260811 14:31:14.216982 3660 heartbeater.cc:347] Connected to a master server at 127.23.186.254:32929
I20260811 14:31:14.217113 3660 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:14.217308 3660 heartbeater.cc:511] Master 127.23.186.254:32929 requested a full tablet report, sending...
I20260811 14:31:14.217707 3280 ts_manager.cc:194] Registered new tserver with Master: 03ba8e78aa9f4adb9c56de4a8b3508e5 (127.23.186.194:43935)
I20260811 14:31:14.222158 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3511
I20260811 14:31:14.222245 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:31:14.222463 24299 external_mini_cluster.cc:1428] Setting key 63edbd64a47b40d2a780fd1b4b88baed
I20260811 14:31:14.223448 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:32929
--builtin_ntp_servers=127.23.186.212:33369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 14:31:14.308888 3666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:14.309139 3666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:14.309180 3666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:14.310438 3666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:14.310510 3666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:14.310564 3666 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:31:14.312005 3666 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:32929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:14.312304 3666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:14.312642 3666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:14.317910 3673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:14.317909 3672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:14.317909 3675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:14.318274 3666 server_base.cc:1034] running on GCE node
I20260811 14:31:14.318531 3666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:14.318815 3666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:14.319969 3666 hybrid_clock.cc:648] HybridClock initialized: now 1786458674319954 us; error 36 us; skew 500 ppm
I20260811 14:31:14.321403 3666 webserver.cc:460] Webserver started at http://127.23.186.195:33971/ using document root <none> and password file <none>
I20260811 14:31:14.321682 3666 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:14.321734 3666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:14.321918 3666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:14.323019 3666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "c8a58641a6d94e1383ffe486cc36f0a5"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "392ffa00a26e0d24db3a4f398123b5ab"
server_key_iv: "ad43e513d6087d87505420e49f0a4436"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.323400 3666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "c8a58641a6d94e1383ffe486cc36f0a5"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "392ffa00a26e0d24db3a4f398123b5ab"
server_key_iv: "ad43e513d6087d87505420e49f0a4436"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.324827 3666 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260811 14:31:14.325631 3684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.325827 3666 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:31:14.325927 3666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "c8a58641a6d94e1383ffe486cc36f0a5"
format_stamp: "Formatted at 2026-08-11 14:31:14 on dist-test-slave-cbsf"
server_key: "392ffa00a26e0d24db3a4f398123b5ab"
server_key_iv: "ad43e513d6087d87505420e49f0a4436"
server_key_version: "encryptionkey@0"
I20260811 14:31:14.326007 3666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:14.329698 3666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:14.329985 3666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:14.330147 3666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:14.330363 3666 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:14.330690 3666 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:14.330724 3666 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.330776 3666 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:14.330806 3666 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:14.337318 3666 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:43761
I20260811 14:31:14.337402 3819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:43761 every 8 connection(s)
I20260811 14:31:14.337760 3666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:31:14.339192 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3666
I20260811 14:31:14.339298 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:31:14.339493 24299 external_mini_cluster.cc:1428] Setting key 1305d02a8844270ef1106513ab099f81
I20260811 14:31:14.341337 3329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:14.338560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53383 (local address 127.23.186.254:32929)
0811 14:31:14.338632 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:14.338634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:14.338861 (+ 227us) server_negotiation.cc:407] Connection header received
0811 14:31:14.339605 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:14.339606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:14.339613 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:14.339676 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:14.341093 (+ 1417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:14.341095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:14.341095 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:31:14.341100 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:14.341118 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:31:14.341122 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:14.341175 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:14.341251 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 14:31:14.341266 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:31:14.341557 3823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:14.338534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:32929 (local address 127.23.186.195:53383)
0811 14:31:14.338742 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:14.338776 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:14.339476 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:14.339779 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:14.339783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:14.339791 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:14.340938 (+ 1147us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:31:14.340939 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:14.340969 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:31:14.340998 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:14.341182 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:14.341187 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:14.341190 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:31:14.341264 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 14:31:14.341327 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":94,"threads_started":1}
I20260811 14:31:14.342041 3821 heartbeater.cc:347] Connected to a master server at 127.23.186.254:32929
I20260811 14:31:14.342119 3821 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:14.342294 3821 heartbeater.cc:511] Master 127.23.186.254:32929 requested a full tablet report, sending...
I20260811 14:31:14.342682 3280 ts_manager.cc:194] Registered new tserver with Master: c8a58641a6d94e1383ffe486cc36f0a5 (127.23.186.195:43761)
I20260811 14:31:14.351490 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 14:31:14.353554 3329 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:31:14.353087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35652 (local address 127.23.186.254:32929)
0811 14:31:14.353170 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:14.353172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:14.353240 (+ 68us) server_negotiation.cc:407] Connection header received
0811 14:31:14.353276 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:14.353276 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:14.353281 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:14.353330 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:14.353477 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35652: BlockingRecv error: recv got EOF from 127.0.0.1:35652 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 14:31:14.353586 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3354
I20260811 14:31:14.362215 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3511
I20260811 14:31:14.372012 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3666
I20260811 14:31:14.378700 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3246
2026-08-11T14:31:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (597 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:31:14 dist-test-slave-cbsf krb5kdc[3836](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:31:14 dist-test-slave-cbsf krb5kdc[3836](info): set up 2 sockets
Aug 11 14:31:14 dist-test-slave-cbsf krb5kdc[3836](info): commencing operation
krb5kdc: starting...
W20260811 14:31:15.517537 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:31:15 dist-test-slave-cbsf krb5kdc[3836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:31:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:31:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:31:15.651228 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38689
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:38689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:31:15.737502 3855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:15.737788 3855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:15.737831 3855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:15.739095 3855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:31:15.739185 3855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:15.739210 3855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:15.739231 3855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:31:15.739262 3855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:31:15.740895 3855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:38689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:15.741230 3855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:15.741494 3855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:15.746855 3863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:15.746888 3860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:15.746867 3861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:15.747124 3855 server_base.cc:1034] running on GCE node
I20260811 14:31:15.747397 3855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:15.747743 3855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:15.748939 3855 hybrid_clock.cc:648] HybridClock initialized: now 1786458675748914 us; error 42 us; skew 500 ppm
Aug 11 14:31:15 dist-test-slave-cbsf krb5kdc[3836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458675, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:31:15.752074 3855 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:31:15.752568 3855 webserver.cc:460] Webserver started at http://127.23.186.254:36199/ using document root <none> and password file <none>
I20260811 14:31:15.752883 3855 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:15.752962 3855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:15.753105 3855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:15.754256 3855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89"
format_stamp: "Formatted at 2026-08-11 14:31:15 on dist-test-slave-cbsf"
server_key: "28fa9c228ff96ba6201dd6799249c429"
server_key_iv: "35d206bf93838714bffca3a69b66ea16"
server_key_version: "encryptionkey@0"
I20260811 14:31:15.754724 3855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89"
format_stamp: "Formatted at 2026-08-11 14:31:15 on dist-test-slave-cbsf"
server_key: "28fa9c228ff96ba6201dd6799249c429"
server_key_iv: "35d206bf93838714bffca3a69b66ea16"
server_key_version: "encryptionkey@0"
I20260811 14:31:15.756390 3855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:15.757447 3871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:15.757661 3855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:31:15.757776 3855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89"
format_stamp: "Formatted at 2026-08-11 14:31:15 on dist-test-slave-cbsf"
server_key: "28fa9c228ff96ba6201dd6799249c429"
server_key_iv: "35d206bf93838714bffca3a69b66ea16"
server_key_version: "encryptionkey@0"
I20260811 14:31:15.757864 3855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:15.769866 3855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:15.772367 3855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:15.772549 3855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:15.776902 3855 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:38689
I20260811 14:31:15.776942 3932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:38689 every 8 connection(s)
I20260811 14:31:15.777351 3855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:31:15.778385 3933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:31:15.781050 3933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89: Bootstrap starting.
I20260811 14:31:15.781723 3933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89: Neither blocks nor log segments found. Creating new log.
I20260811 14:31:15.782055 3933 log.cc:824] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89: Log is configured to *not* fsync() on all Append() calls
I20260811 14:31:15.782847 3933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89: No bootstrap required, opened a new log
I20260811 14:31:15.784523 3933 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38689 } }
I20260811 14:31:15.784741 3933 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:31:15.784802 3933 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1fd48030aba4ce09ad0f7ff1fb92f89, State: Initialized, Role: FOLLOWER
I20260811 14:31:15.784965 3933 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38689 } }
I20260811 14:31:15.785032 3933 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:31:15.785079 3933 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:31:15.785130 3933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:31:15.786044 3933 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38689 } }
I20260811 14:31:15.786168 3933 leader_election.cc:302] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1fd48030aba4ce09ad0f7ff1fb92f89; no voters:
I20260811 14:31:15.786362 3933 leader_election.cc:288] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:31:15.786432 3936 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:31:15.786617 3936 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [term 1 LEADER]: Becoming Leader. State: Replica: d1fd48030aba4ce09ad0f7ff1fb92f89, State: Running, Role: LEADER
I20260811 14:31:15.786669 3933 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:31:15.786715 3936 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38689 } }
I20260811 14:31:15.787182 3937 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38689 } } }
I20260811 14:31:15.787292 3937 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [sys.catalog]: This master's current role is: LEADER
I20260811 14:31:15.787178 3938 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1fd48030aba4ce09ad0f7ff1fb92f89. Latest consensus state: current_term: 1 leader_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1fd48030aba4ce09ad0f7ff1fb92f89" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38689 } } }
I20260811 14:31:15.787365 3938 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89 [sys.catalog]: This master's current role is: LEADER
I20260811 14:31:15.787509 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3855
I20260811 14:31:15.787631 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:31:15.787672 3944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:31:15.787839 24299 external_mini_cluster.cc:1428] Setting key 02d0b608a5d3418c0a37fc53b863ee03
I20260811 14:31:15.788249 3944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:31:15.790086 3944 catalog_manager.cc:1347] Generated new cluster ID: a8f9b3c836844ad19184ebdc9b4c6056
I20260811 14:31:15.790143 3944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:31:15 dist-test-slave-cbsf krb5kdc[3836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:31:15.800827 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:15.788396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.23.186.254:38689)
0811 14:31:15.788564 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:15.788573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:15.788609 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:31:15.789141 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:15.789151 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:15.789217 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:15.789368 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:15.790183 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:15.791587 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:15.792730 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:15.793394 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:15.793514 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:15.796552 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:15.796559 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:15.796559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:31:15.796579 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:15.798867 (+ 2288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:15.799309 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:15.799314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:15.799315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:15.799457 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:15.799747 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:15.799748 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:15.799749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:15.800117 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:15.800281 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:15.800381 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 14:31:15.800546 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":74,"threads_started":1}
I20260811 14:31:15.802865 3944 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:31:15.803395 3944 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:31:15.805992 3944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89: Generated new TSK 0
I20260811 14:31:15.806205 3944 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:31:15.833943 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38689
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:31:15.922613 3964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:15.922855 3964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:15.922883 3964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:15.924140 3964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:15.924192 3964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:15.924222 3964 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:31:15.925884 3964 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3964
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:15.926172 3964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:15.926432 3964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:15.931793 3970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:15.931917 3964 server_base.cc:1034] running on GCE node
W20260811 14:31:15.931814 3973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:15.931784 3971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:15.932215 3964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:15.932474 3964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:15.933642 3964 hybrid_clock.cc:648] HybridClock initialized: now 1786458675933617 us; error 60 us; skew 500 ppm
Aug 11 14:31:15 dist-test-slave-cbsf krb5kdc[3836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458675, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:31:15.936233 3964 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:31:15.936753 3964 webserver.cc:460] Webserver started at http://127.23.186.193:44259/ using document root <none> and password file <none>
I20260811 14:31:15.937016 3964 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:15.937096 3964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:15.937260 3964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:15.938381 3964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "664fecc3335e47f7a859bcb0bc662ae2"
format_stamp: "Formatted at 2026-08-11 14:31:15 on dist-test-slave-cbsf"
server_key: "434e6db1de3e128f2d82aa31c4f5bd39"
server_key_iv: "aad769e576350992a122479edc7245e9"
server_key_version: "encryptionkey@0"
I20260811 14:31:15.938755 3964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "664fecc3335e47f7a859bcb0bc662ae2"
format_stamp: "Formatted at 2026-08-11 14:31:15 on dist-test-slave-cbsf"
server_key: "434e6db1de3e128f2d82aa31c4f5bd39"
server_key_iv: "aad769e576350992a122479edc7245e9"
server_key_version: "encryptionkey@0"
I20260811 14:31:15.940165 3964 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:15.941016 3981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:15.941210 3964 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:31:15.941320 3964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "664fecc3335e47f7a859bcb0bc662ae2"
format_stamp: "Formatted at 2026-08-11 14:31:15 on dist-test-slave-cbsf"
server_key: "434e6db1de3e128f2d82aa31c4f5bd39"
server_key_iv: "aad769e576350992a122479edc7245e9"
server_key_version: "encryptionkey@0"
I20260811 14:31:15.941398 3964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:15.971807 3964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:15.974179 3964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:15.974377 3964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:15.974664 3964 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:15.975016 3964 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:15.975081 3964 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:15.975123 3964 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:15.975152 3964 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:15.981457 3964 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:41709
I20260811 14:31:15.981510 4115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:41709 every 8 connection(s)
I20260811 14:31:15.981858 3964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
Aug 11 14:31:15 dist-test-slave-cbsf krb5kdc[3836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458675, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:31:15.990835 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 3964
I20260811 14:31:15.990954 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:31:15.991166 24299 external_mini_cluster.cc:1428] Setting key 6964479bf41438a507a8801beedf9713
I20260811 14:31:15.992177 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:15.982695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:39089 (local address 127.23.186.254:38689)
0811 14:31:15.982787 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:15.982789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:15.982960 (+ 171us) server_negotiation.cc:407] Connection header received
0811 14:31:15.983480 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:15.983481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:15.983496 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:15.983554 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:15.984579 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:15.985178 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:15.986250 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:15.986520 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:15.986578 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:15.989003 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:15.989004 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:15.989004 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:31:15.989012 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:15.990506 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:15.990981 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:15.990983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:15.990983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:15.991064 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:15.991404 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:15.991406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:15.991406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:15.991600 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:15.991674 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:15.991941 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 14:31:15.992055 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 14:31:15.992266 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:15.982629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38689 (local address 127.23.186.193:39089)
0811 14:31:15.982833 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:15.982873 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:15.983380 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:15.983608 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:15.983612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:15.984229 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:15.984481 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:15.984486 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:15.985279 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:15.985280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:15.986154 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:15.986157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:15.986598 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:15.986599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:15.986730 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:15.987232 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:15.987240 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:15.988902 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:15.990593 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:31:15.990600 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:31:15.990603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:31:15.990854 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:31:15.991133 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:31:15.991134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:31:15.991135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:31:15.991292 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:31:15.991671 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:15.991678 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:15.991819 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 14:31:15.991913 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 14:31:15.992060 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":111,"threads_started":1}
I20260811 14:31:15.992813 4116 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38689
I20260811 14:31:15.992940 4116 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:15.993157 4116 heartbeater.cc:511] Master 127.23.186.254:38689 requested a full tablet report, sending...
I20260811 14:31:15.993775 3892 ts_manager.cc:194] Registered new tserver with Master: 664fecc3335e47f7a859bcb0bc662ae2 (127.23.186.193:41709)
I20260811 14:31:15.994462 3892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:39089
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:31:16.020612 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38689
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:31:16.105702 4121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:16.105994 4121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:16.106038 4121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:16.107266 4121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:16.107353 4121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:16.107402 4121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:31:16.108994 4121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:16.109315 4121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:16.109573 4121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:16.115181 4127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:16.115191 4121 server_base.cc:1034] running on GCE node
W20260811 14:31:16.115173 4126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:16.115173 4129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:16.115625 4121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:16.115942 4121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:16.117139 4121 hybrid_clock.cc:648] HybridClock initialized: now 1786458676117133 us; error 38 us; skew 500 ppm
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[3836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458676, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:31:16.119783 4121 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:31:16.120308 4121 webserver.cc:460] Webserver started at http://127.23.186.194:43461/ using document root <none> and password file <none>
I20260811 14:31:16.120556 4121 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:16.120628 4121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:16.120816 4121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:16.121929 4121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "c358561a280347d297651ded7ce0b203"
format_stamp: "Formatted at 2026-08-11 14:31:16 on dist-test-slave-cbsf"
server_key: "4483b56b449e0ccac9d2643944cd2696"
server_key_iv: "94beb3f23b7afb9888cd614995960767"
server_key_version: "encryptionkey@0"
I20260811 14:31:16.122310 4121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "c358561a280347d297651ded7ce0b203"
format_stamp: "Formatted at 2026-08-11 14:31:16 on dist-test-slave-cbsf"
server_key: "4483b56b449e0ccac9d2643944cd2696"
server_key_iv: "94beb3f23b7afb9888cd614995960767"
server_key_version: "encryptionkey@0"
I20260811 14:31:16.123726 4121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:16.124507 4140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:16.124684 4121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:31:16.124843 4121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "c358561a280347d297651ded7ce0b203"
format_stamp: "Formatted at 2026-08-11 14:31:16 on dist-test-slave-cbsf"
server_key: "4483b56b449e0ccac9d2643944cd2696"
server_key_iv: "94beb3f23b7afb9888cd614995960767"
server_key_version: "encryptionkey@0"
I20260811 14:31:16.124943 4121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:16.134111 4121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:16.136010 4121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:16.136183 4121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:16.136445 4121 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:16.136849 4121 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:16.136904 4121 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:16.136947 4121 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:16.136974 4121 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:16.143090 4121 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:35213
I20260811 14:31:16.143146 4268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:35213 every 8 connection(s)
I20260811 14:31:16.143538 4121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:31:16.146906 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 4121
I20260811 14:31:16.147001 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:31:16.147204 24299 external_mini_cluster.cc:1428] Setting key 6ea99f416eb426e0e3f84e136ee70cbc
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[3836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458676, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:31:16.154508 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.144375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:34185 (local address 127.23.186.254:38689)
0811 14:31:16.144446 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.144448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.144677 (+ 229us) server_negotiation.cc:407] Connection header received
0811 14:31:16.145258 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.145259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.145272 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.145323 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:16.146397 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.147091 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.148336 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.148638 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.148758 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.151280 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:16.151281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:16.151282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:31:16.151291 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:16.153058 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:16.153462 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:16.153464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:16.153464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:16.153530 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:16.153764 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:16.153765 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:16.153765 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:16.153931 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:16.154014 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.154269 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.154367 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:31:16.154521 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.144358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38689 (local address 127.23.186.194:34185)
0811 14:31:16.144544 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.144595 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.145138 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.145444 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.145448 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.146085 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:16.146292 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.146297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.147200 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.147201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.148252 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.148255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.148820 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.148821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.148926 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.149502 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:16.149512 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:16.151185 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:16.153152 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:31:16.153159 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:31:16.153161 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:31:16.153385 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:31:16.153600 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:31:16.153600 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:31:16.153601 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:31:16.153699 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:31:16.154003 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:16.154008 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:16.154134 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.154218 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.154346 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260811 14:31:16.155190 4269 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38689
I20260811 14:31:16.155318 4269 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:16.155555 4269 heartbeater.cc:511] Master 127.23.186.254:38689 requested a full tablet report, sending...
I20260811 14:31:16.156023 3892 ts_manager.cc:194] Registered new tserver with Master: c358561a280347d297651ded7ce0b203 (127.23.186.194:35213)
I20260811 14:31:16.156424 3892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:34185
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:31:16.177685 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38689
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:31:16.266043 4275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:31:16.266302 4275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:31:16.266331 4275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:31:16.267618 4275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:31:16.267670 4275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:31:16.267701 4275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:31:16.269400 4275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:31:16.269690 4275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:31:16.269938 4275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:31:16.275305 4281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:31:16.275326 4282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:16.275401 4275 server_base.cc:1034] running on GCE node
W20260811 14:31:16.275326 4284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:31:16.275766 4275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:31:16.276041 4275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:31:16.277230 4275 hybrid_clock.cc:648] HybridClock initialized: now 1786458676277222 us; error 37 us; skew 500 ppm
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[3836](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458676, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:31:16.280107 4275 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:31:16.280619 4275 webserver.cc:460] Webserver started at http://127.23.186.195:33123/ using document root <none> and password file <none>
I20260811 14:31:16.280889 4275 fs_manager.cc:359] Metadata directory not provided
I20260811 14:31:16.280965 4275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:31:16.281132 4275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:31:16.282258 4275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "c44371b1f9ca497585881b13697ca897"
format_stamp: "Formatted at 2026-08-11 14:31:16 on dist-test-slave-cbsf"
server_key: "53e72c06ce8812b045689c273dfb557c"
server_key_iv: "04c0ef6c2c799935c5dd30dfc5a120ec"
server_key_version: "encryptionkey@0"
I20260811 14:31:16.282657 4275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "c44371b1f9ca497585881b13697ca897"
format_stamp: "Formatted at 2026-08-11 14:31:16 on dist-test-slave-cbsf"
server_key: "53e72c06ce8812b045689c273dfb557c"
server_key_iv: "04c0ef6c2c799935c5dd30dfc5a120ec"
server_key_version: "encryptionkey@0"
I20260811 14:31:16.284183 4275 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:31:16.285025 4293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:16.285208 4275 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:31:16.285322 4275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "c44371b1f9ca497585881b13697ca897"
format_stamp: "Formatted at 2026-08-11 14:31:16 on dist-test-slave-cbsf"
server_key: "53e72c06ce8812b045689c273dfb557c"
server_key_iv: "04c0ef6c2c799935c5dd30dfc5a120ec"
server_key_version: "encryptionkey@0"
I20260811 14:31:16.285401 4275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:31:16.302788 4275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:31:16.305445 4275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:31:16.305625 4275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:31:16.305915 4275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:31:16.306334 4275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:31:16.306392 4275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:16.306471 4275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:31:16.306502 4275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:31:16.312924 4275 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:32793
I20260811 14:31:16.313045 4421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:32793 every 8 connection(s)
I20260811 14:31:16.313357 4275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:31:16.314214 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 4275
I20260811 14:31:16.314329 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:31:16.314510 24299 external_mini_cluster.cc:1428] Setting key 79cd062ce4a2389a6f42b60d17d17f56
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[3836](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458676, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:31:16.323454 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.314155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:60857 (local address 127.23.186.254:38689)
0811 14:31:16.314231 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.314233 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.314863 (+ 630us) server_negotiation.cc:407] Connection header received
0811 14:31:16.315581 (+ 718us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.315582 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.315594 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.315642 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:16.316604 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.317202 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.318230 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.318434 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.318492 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.320881 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:16.320882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:16.320883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:31:16.320891 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:16.322175 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:16.322562 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:16.322563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:16.322563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:16.322617 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:16.322842 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:16.322843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:16.322843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:16.322965 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:16.323021 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.323276 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.323354 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:31:16.323633 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.314111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38689 (local address 127.23.186.195:60857)
0811 14:31:16.314706 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.314776 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.315473 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.315698 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.315701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.316315 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:31:16.316524 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.316528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.317289 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.317290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.318147 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.318149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.318514 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.318515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.318623 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.319206 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:31:16.319215 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:31:16.320763 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:31:16.322254 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:31:16.322261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:31:16.322263 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:31:16.322491 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:31:16.322678 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:31:16.322679 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:31:16.322679 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:31:16.322781 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:31:16.323024 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:31:16.323029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:31:16.323156 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.323241 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.323385 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":92,"threads_started":1}
I20260811 14:31:16.324156 4422 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38689
I20260811 14:31:16.324270 4422 heartbeater.cc:464] Registering TS with master...
I20260811 14:31:16.324489 4422 heartbeater.cc:511] Master 127.23.186.254:38689 requested a full tablet report, sending...
I20260811 14:31:16.324956 3892 ts_manager.cc:194] Registered new tserver with Master: c44371b1f9ca497585881b13697ca897 (127.23.186.195:32793)
I20260811 14:31:16.325338 3892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:60857
I20260811 14:31:16.326431 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:31:16.333910 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.327758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48318 (local address 127.23.186.254:38689)
0811 14:31:16.327819 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.327821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.327948 (+ 127us) server_negotiation.cc:407] Connection header received
0811 14:31:16.327995 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.327995 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.328004 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.328060 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:31:16.328774 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.329338 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.330075 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.330261 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.330316 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.331600 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:31:16.331601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:31:16.331602 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:31:16.331607 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:31:16.332921 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:16.333244 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:16.333245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:16.333245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:16.333296 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:31:16.333460 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:31:16.333460 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:31:16.333460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:31:16.333622 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:31:16.333700 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.333786 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.333843 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:31:16.335530 3892 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:31:16.337317 3892 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:31:16.344563 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.340446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.23.186.194:35213)
0811 14:31:16.340642 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.340648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.340659 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:31:16.340719 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.340721 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.340738 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.340968 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.341045 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.342041 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.343431 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.344144 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.344217 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.344269 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.344408 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.344455 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
I20260811 14:31:16.344594 4440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.340220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:41709 (local address 127.0.0.1:59834)
0811 14:31:16.340468 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.340479 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.340539 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.341038 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.341039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.341040 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.341098 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.341099 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.341980 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.341981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.343122 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.343125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.344205 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.344206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.344292 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.344307 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.344382 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.344455 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":70,"thread_start_us":76,"threads_started":1}
I20260811 14:31:16.344607 4439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.340022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:35213 (local address 127.0.0.1:53794)
0811 14:31:16.340148 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.340158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.340215 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.340848 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.340850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.340852 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.340893 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.340895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.342135 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.342136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.343319 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.343322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.344233 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.344234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.344298 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.344319 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.344386 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.344483 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"mutex_wait_us":51,"thread_start_us":87,"threads_started":1}
I20260811 14:31:16.344606 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.339787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:32793 (local address 127.0.0.1:46070)
0811 14:31:16.339915 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.339965 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.340029 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.340329 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.340332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.340335 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.340408 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.340410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.341928 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.341937 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.343448 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.343450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.344359 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.344359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.344419 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.344422 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.344478 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.344516 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":58,"thread_start_us":80,"threads_started":1}
I20260811 14:31:16.344969 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.339889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46070 (local address 127.23.186.195:32793)
0811 14:31:16.340065 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.340070 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.340082 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:31:16.340127 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.340129 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.340148 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.340237 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.340605 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.341803 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.343532 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.344171 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.344267 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.344325 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.344646 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.344874 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":108,"threads_started":1}
I20260811 14:31:16.345476 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.340347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59834 (local address 127.23.186.193:41709)
0811 14:31:16.340795 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.340800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.340810 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:16.340851 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.340854 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.340871 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.340958 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.341177 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.341866 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.343215 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.344010 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.344100 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.344165 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.345343 (+ 1178us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.345392 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":101,"threads_started":1}
I20260811 14:31:16.345619 4193 tablet_service.cc:1467] Processing CreateTablet for tablet 02764ae616b7405d90f3e62eaa9b4434 (DEFAULT_TABLE table=test-table [id=8d07bf598b9742a98987218510ef88de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:31:16.345618 4350 tablet_service.cc:1467] Processing CreateTablet for tablet 02764ae616b7405d90f3e62eaa9b4434 (DEFAULT_TABLE table=test-table [id=8d07bf598b9742a98987218510ef88de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:31:16.346004 4350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02764ae616b7405d90f3e62eaa9b4434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:31:16.346009 4193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02764ae616b7405d90f3e62eaa9b4434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:31:16.346182 4035 tablet_service.cc:1467] Processing CreateTablet for tablet 02764ae616b7405d90f3e62eaa9b4434 (DEFAULT_TABLE table=test-table [id=8d07bf598b9742a98987218510ef88de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:31:16.346505 4035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02764ae616b7405d90f3e62eaa9b4434. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:31:16.348783 4445 tablet_bootstrap.cc:492] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Bootstrap starting.
I20260811 14:31:16.349496 4445 tablet_bootstrap.cc:654] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Neither blocks nor log segments found. Creating new log.
I20260811 14:31:16.349563 4446 tablet_bootstrap.cc:492] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Bootstrap starting.
I20260811 14:31:16.350147 4447 tablet_bootstrap.cc:492] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: Bootstrap starting.
I20260811 14:31:16.350229 4446 tablet_bootstrap.cc:654] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Neither blocks nor log segments found. Creating new log.
I20260811 14:31:16.350644 4445 log.cc:824] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Log is configured to *not* fsync() on all Append() calls
I20260811 14:31:16.350689 4447 tablet_bootstrap.cc:654] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: Neither blocks nor log segments found. Creating new log.
I20260811 14:31:16.350847 4446 log.cc:824] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Log is configured to *not* fsync() on all Append() calls
I20260811 14:31:16.351354 4447 log.cc:824] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: Log is configured to *not* fsync() on all Append() calls
I20260811 14:31:16.351673 4445 tablet_bootstrap.cc:492] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: No bootstrap required, opened a new log
I20260811 14:31:16.351758 4445 ts_tablet_manager.cc:1397] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:31:16.352104 4447 tablet_bootstrap.cc:492] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: No bootstrap required, opened a new log
I20260811 14:31:16.352187 4447 ts_tablet_manager.cc:1397] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:31:16.353124 4446 tablet_bootstrap.cc:492] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: No bootstrap required, opened a new log
I20260811 14:31:16.353201 4446 ts_tablet_manager.cc:1397] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 14:31:16.353605 4445 raft_consensus.cc:348] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.353789 4445 raft_consensus.cc:374] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:31:16.353823 4445 raft_consensus.cc:729] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c44371b1f9ca497585881b13697ca897, State: Initialized, Role: FOLLOWER
I20260811 14:31:16.353936 4445 consensus_queue.cc:260] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.353899 4447 raft_consensus.cc:348] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.354082 4447 raft_consensus.cc:374] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:31:16.354120 4447 raft_consensus.cc:729] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 664fecc3335e47f7a859bcb0bc662ae2, State: Initialized, Role: FOLLOWER
I20260811 14:31:16.354256 4445 ts_tablet_manager.cc:1428] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 14:31:16.354234 4447 consensus_queue.cc:260] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.354457 4422 heartbeater.cc:503] Master 127.23.186.254:38689 was elected leader, sending a full tablet report...
I20260811 14:31:16.354532 4447 ts_tablet_manager.cc:1428] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:31:16.354691 4116 heartbeater.cc:503] Master 127.23.186.254:38689 was elected leader, sending a full tablet report...
I20260811 14:31:16.355352 4446 raft_consensus.cc:348] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.355521 4446 raft_consensus.cc:374] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:31:16.355571 4446 raft_consensus.cc:729] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c358561a280347d297651ded7ce0b203, State: Initialized, Role: FOLLOWER
I20260811 14:31:16.355677 4446 consensus_queue.cc:260] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.356019 4446 ts_tablet_manager.cc:1428] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:31:16.356141 4269 heartbeater.cc:503] Master 127.23.186.254:38689 was elected leader, sending a full tablet report...
W20260811 14:31:16.394201 4270 tablet.cc:2369] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:31:16.452802 4452 raft_consensus.cc:482] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:31:16.452873 4453 raft_consensus.cc:482] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:31:16.452957 4452 raft_consensus.cc:504] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.453014 4453 raft_consensus.cc:504] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.453428 4453 leader_election.cc:288] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c44371b1f9ca497585881b13697ca897 (127.23.186.195:32793), c358561a280347d297651ded7ce0b203 (127.23.186.194:35213)
I20260811 14:31:16.453428 4452 leader_election.cc:288] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c358561a280347d297651ded7ce0b203 (127.23.186.194:35213), 664fecc3335e47f7a859bcb0bc662ae2 (127.23.186.193:41709)
I20260811 14:31:16.457543 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.453636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:32793 (local address 127.23.186.193:46973)
0811 14:31:16.453916 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.453927 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.453968 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.454192 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.454193 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.454195 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.454265 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.454267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455210 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.455211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.456485 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.456488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457270 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.457351 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457353 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.457403 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.457447 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":58}
I20260811 14:31:16.457635 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.454035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:34017 (local address 127.23.186.193:41709)
0811 14:31:16.454073 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.454074 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.454081 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:31:16.454111 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.454112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.454121 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.454168 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.454367 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455114 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.456596 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457168 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457218 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457229 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.457434 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.457460 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:31:16.457656 4424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.453650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:35213 (local address 127.23.186.195:43063)
0811 14:31:16.453699 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.453712 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.453760 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.454243 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.454244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.454246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.454340 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.454343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455270 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.455271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.456486 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.456489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457395 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.457467 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457471 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.457519 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.457557 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 14:31:16.457803 4456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.453781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:41709 (local address 127.23.186.195:34017)
0811 14:31:16.453961 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.453971 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.454017 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.454215 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.454216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.454217 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.454285 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.454287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455210 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.455211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.456484 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.456488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457286 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.457351 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457353 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.457488 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.457526 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260811 14:31:16.457856 4458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.454096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:58983 (local address 127.23.186.194:35213)
0811 14:31:16.454282 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.454285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.454293 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:31:16.454315 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.454316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.454326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.454578 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.454623 (+ 45us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455292 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.456662 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457290 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457600 (+ 310us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457608 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.457627 (+ 19us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.457662 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
I20260811 14:31:16.458088 4060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "c44371b1f9ca497585881b13697ca897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "664fecc3335e47f7a859bcb0bc662ae2" is_pre_election: true
I20260811 14:31:16.458159 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.453808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:43063 (local address 127.23.186.194:35213)
0811 14:31:16.453861 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.453863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.453874 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:31:16.453908 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.453909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.453921 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.453986 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.454423 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455181 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.456595 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457277 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457352 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457362 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.458074 (+ 712us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.458111 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":25}
I20260811 14:31:16.458220 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.453750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:46973 (local address 127.23.186.195:32793)
0811 14:31:16.454052 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.454054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.454063 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:31:16.454089 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.454090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.454098 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.454147 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:31:16.454344 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455109 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.456587 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457166 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457233 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457241 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.458135 (+ 894us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.458172 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281}
I20260811 14:31:16.458220 4060 raft_consensus.cc:2454] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c44371b1f9ca497585881b13697ca897 in term 0.
I20260811 14:31:16.458366 4455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.453776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:35213 (local address 127.23.186.193:58983)
0811 14:31:16.454110 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.454123 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:31:16.454172 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:31:16.454422 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:31:16.454423 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:31:16.454425 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:31:16.454491 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.454492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.455378 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.455379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.456573 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:31:16.456577 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.457383 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.457385 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:31:16.457458 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.457462 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:31:16.457495 (+ 33us) client_negotiation.cc:240] Negotiation successful
0811 14:31:16.457551 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":2,"thread_start_us":99,"threads_started":1}
I20260811 14:31:16.458499 4296 leader_election.cc:302] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 664fecc3335e47f7a859bcb0bc662ae2, c44371b1f9ca497585881b13697ca897; no voters:
I20260811 14:31:16.458527 4373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "664fecc3335e47f7a859bcb0bc662ae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c44371b1f9ca497585881b13697ca897" is_pre_election: true
I20260811 14:31:16.458628 4373 raft_consensus.cc:2454] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 664fecc3335e47f7a859bcb0bc662ae2 in term 0.
I20260811 14:31:16.458724 4216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "664fecc3335e47f7a859bcb0bc662ae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c358561a280347d297651ded7ce0b203" is_pre_election: true
I20260811 14:31:16.458755 4217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "c44371b1f9ca497585881b13697ca897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c358561a280347d297651ded7ce0b203" is_pre_election: true
I20260811 14:31:16.458827 3984 leader_election.cc:302] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 664fecc3335e47f7a859bcb0bc662ae2, c44371b1f9ca497585881b13697ca897; no voters:
I20260811 14:31:16.458884 4216 raft_consensus.cc:2454] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 664fecc3335e47f7a859bcb0bc662ae2 in term 0.
I20260811 14:31:16.458673 4452 raft_consensus.cc:2781] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:31:16.459000 4452 raft_consensus.cc:482] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:31:16.459034 4452 raft_consensus.cc:3037] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:31:16.459013 4453 raft_consensus.cc:2781] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:31:16.459216 4453 raft_consensus.cc:482] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:31:16.459245 4453 raft_consensus.cc:3037] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:31:16.459849 4452 raft_consensus.cc:504] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.459940 4453 raft_consensus.cc:504] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.459995 4452 leader_election.cc:288] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [CANDIDATE]: Term 1 election: Requested vote from peers c358561a280347d297651ded7ce0b203 (127.23.186.194:35213), 664fecc3335e47f7a859bcb0bc662ae2 (127.23.186.193:41709)
I20260811 14:31:16.460131 4217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "c44371b1f9ca497585881b13697ca897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c358561a280347d297651ded7ce0b203"
I20260811 14:31:16.460202 4373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "664fecc3335e47f7a859bcb0bc662ae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c44371b1f9ca497585881b13697ca897"
I20260811 14:31:16.460201 4060 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "c44371b1f9ca497585881b13697ca897" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "664fecc3335e47f7a859bcb0bc662ae2"
I20260811 14:31:16.460237 4217 raft_consensus.cc:3037] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:31:16.460283 4060 raft_consensus.cc:2379] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c44371b1f9ca497585881b13697ca897 in current term 1: Already voted for candidate 664fecc3335e47f7a859bcb0bc662ae2 in this term.
I20260811 14:31:16.460283 4373 raft_consensus.cc:2379] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 664fecc3335e47f7a859bcb0bc662ae2 in current term 1: Already voted for candidate c44371b1f9ca497585881b13697ca897 in this term.
I20260811 14:31:16.460285 4216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02764ae616b7405d90f3e62eaa9b4434" candidate_uuid: "664fecc3335e47f7a859bcb0bc662ae2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c358561a280347d297651ded7ce0b203"
I20260811 14:31:16.460525 3984 leader_election.cc:302] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 664fecc3335e47f7a859bcb0bc662ae2; no voters: c358561a280347d297651ded7ce0b203, c44371b1f9ca497585881b13697ca897
I20260811 14:31:16.460762 4453 leader_election.cc:288] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [CANDIDATE]: Term 1 election: Requested vote from peers c44371b1f9ca497585881b13697ca897 (127.23.186.195:32793), c358561a280347d297651ded7ce0b203 (127.23.186.194:35213)
I20260811 14:31:16.460831 4460 raft_consensus.cc:2726] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 14:31:16.461203 4217 raft_consensus.cc:2454] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c44371b1f9ca497585881b13697ca897 in term 1.
I20260811 14:31:16.461380 4296 leader_election.cc:302] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c358561a280347d297651ded7ce0b203, c44371b1f9ca497585881b13697ca897; no voters: 664fecc3335e47f7a859bcb0bc662ae2
I20260811 14:31:16.461470 4452 raft_consensus.cc:2781] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:31:16.461656 4452 raft_consensus.cc:686] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 1 LEADER]: Becoming Leader. State: Replica: c44371b1f9ca497585881b13697ca897, State: Running, Role: LEADER
I20260811 14:31:16.461776 4452 consensus_queue.cc:237] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } }
I20260811 14:31:16.462658 3891 catalog_manager.cc:5495] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 reported cstate change: term changed from 0 to 1, leader changed from <none> to c44371b1f9ca497585881b13697ca897 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "c44371b1f9ca497585881b13697ca897" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c44371b1f9ca497585881b13697ca897" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 32793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 } health_report { overall_health: UNKNOWN } } }
W20260811 14:31:16.482678 4117 tablet.cc:2369] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:31:16.492352 4437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.489843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46084 (local address 127.23.186.195:32793)
0811 14:31:16.489922 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0811 14:31:16.489925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:31:16.489935 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:31:16.489984 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:31:16.489985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:31:16.490002 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:31:16.490080 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:31:16.490242 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.490731 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.491507 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:31:16.491707 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:31:16.491784 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:31:16.491969 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:31:16.492081 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:31:16.492174 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:31:16.492200 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 14:31:16.492233 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:31:16.494324 4217 raft_consensus.cc:1261] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 1 FOLLOWER]: Refusing update from remote peer c44371b1f9ca497585881b13697ca897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:31:16.494351 4060 raft_consensus.cc:1261] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 1 FOLLOWER]: Refusing update from remote peer c44371b1f9ca497585881b13697ca897: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:31:16.494656 4462 consensus_queue.cc:1035] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c358561a280347d297651ded7ce0b203" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 35213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:31:16.494810 4452 consensus_queue.cc:1035] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [LEADER]: Connected to new peer: Peer: permanent_uuid: "664fecc3335e47f7a859bcb0bc662ae2" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:31:16.500114 3892 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:31:16.500233 3892 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:31:16.501062 3892 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d1fd48030aba4ce09ad0f7ff1fb92f89: Sending DeleteTablet for 3 replicas of tablet 02764ae616b7405d90f3e62eaa9b4434
I20260811 14:31:16.501427 4193 tablet_service.cc:1514] Processing DeleteTablet for tablet 02764ae616b7405d90f3e62eaa9b4434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:31:16 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:53794
I20260811 14:31:16.501581 4350 tablet_service.cc:1514] Processing DeleteTablet for tablet 02764ae616b7405d90f3e62eaa9b4434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:31:16 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:46070
I20260811 14:31:16.501716 4477 tablet_replica.cc:330] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: stopping tablet replica
I20260811 14:31:16.501724 4035 tablet_service.cc:1514] Processing DeleteTablet for tablet 02764ae616b7405d90f3e62eaa9b4434 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:31:16 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:59834
I20260811 14:31:16.501830 4477 raft_consensus.cc:2229] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:31:16.501952 4478 tablet_replica.cc:330] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: stopping tablet replica
I20260811 14:31:16.501958 4477 raft_consensus.cc:2258] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:31:16.502038 4478 raft_consensus.cc:2229] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:31:16.502125 4478 raft_consensus.cc:2258] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:31:16.502254 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3964
I20260811 14:31:16.502349 4479 tablet_replica.cc:330] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2: stopping tablet replica
I20260811 14:31:16.502444 4479 raft_consensus.cc:2229] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:31:16.502491 4477 ts_tablet_manager.cc:1905] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:31:16.502534 4479 raft_consensus.cc:2258] T 02764ae616b7405d90f3e62eaa9b4434 P 664fecc3335e47f7a859bcb0bc662ae2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:31:16.503252 4478 ts_tablet_manager.cc:1905] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:31:16.505108 4478 ts_tablet_manager.cc:1918] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:31:16.505172 4478 log.cc:1196] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/02764ae616b7405d90f3e62eaa9b4434
I20260811 14:31:16.505453 4478 ts_tablet_manager.cc:1939] T 02764ae616b7405d90f3e62eaa9b4434 P c44371b1f9ca497585881b13697ca897: Deleting consensus metadata
I20260811 14:31:16.505623 4477 ts_tablet_manager.cc:1918] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:31:16.505690 4477 log.cc:1196] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/02764ae616b7405d90f3e62eaa9b4434
I20260811 14:31:16.505967 3873 catalog_manager.cc:4841] TS c44371b1f9ca497585881b13697ca897 (127.23.186.195:32793): tablet 02764ae616b7405d90f3e62eaa9b4434 (table test-table [id=8d07bf598b9742a98987218510ef88de]) successfully deleted
I20260811 14:31:16.506361 4477 ts_tablet_manager.cc:1939] T 02764ae616b7405d90f3e62eaa9b4434 P c358561a280347d297651ded7ce0b203: Deleting consensus metadata
I20260811 14:31:16.506706 3873 catalog_manager.cc:4841] TS c358561a280347d297651ded7ce0b203 (127.23.186.194:35213): tablet 02764ae616b7405d90f3e62eaa9b4434 (table test-table [id=8d07bf598b9742a98987218510ef88de]) successfully deleted
W20260811 14:31:16.512404 3873 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:41709 (error 108)
I20260811 14:31:16.512674 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 4121
I20260811 14:31:16.514513 4436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:31:16.512934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:41709 (local address 127.0.0.1:59842)
0811 14:31:16.514415 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0811 14:31:16.514471 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:41709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1449}
W20260811 14:31:16.514823 3873 catalog_manager.cc:4584] TS 664fecc3335e47f7a859bcb0bc662ae2 (127.23.186.193:41709): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02764ae616b7405d90f3e62eaa9b4434: Network error: Client connection negotiation failed: client connection to 127.23.186.193:41709: connect: Connection refused (error 111)
I20260811 14:31:16.522456 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 4275
I20260811 14:31:16.530686 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 3855
2026-08-11T14:31:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2153 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[4485](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[4485](info): set up 2 sockets
Aug 11 14:31:16 dist-test-slave-cbsf krb5kdc[4485](info): commencing operation
krb5kdc: starting...
W20260811 14:31:17.669566 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:31:17 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:31:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:31:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/rangeradmin_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/rangeradmin_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/rangerlookup_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/rangerlookup_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
I20260811 14:31:17.855792 24299 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLI13IK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/postgres -l logfile start
2026-08-11 14:31:20.472 UTC [4520] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 14:31:20.473 UTC [4520] LOG: listening on IPv4 address "127.23.186.212", port 40777
2026-08-11 14:31:20.477 UTC [4520] LOG: listening on Unix socket "/tmp/.s.PGSQL.40777"
2026-08-11 14:31:20.500 UTC [4523] LOG: database system was shut down at 2026-08-11 14:31:19 UTC
2026-08-11 14:31:20.504 UTC [4520] LOG: database system is ready to accept connections
I20260811 14:31:21.549858 24299 mini_postgres.cc:96] Postgres bound to 40777
2026-08-11 14:31:21.558 UTC [4531] FATAL: database "slave" does not exist
127.23.186.212:40777 - accepting connections
I20260811 14:31:21.559201 24299 mini_ranger.cc:161] Starting Ranger...
I20260811 14:31:21.576234 24299 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 14:31:22.065762 24299 mini_ranger.cc:87] Created ranger Postgres database
I20260811 14:31:22.065858 24299 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-admin
2026-08-11 14:31:22,398 [I] DB FLAVOR :POSTGRES
2026-08-11 14:31:22,398 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:31:22,399 [I] Checking connection..
2026-08-11 14:31:22,399 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 14:31:22,678 [I] Checking connection passed.
2026-08-11 14:31:22,678 [I] --------- Verifying version history table ---------
2026-08-11 14:31:22,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:31:22,942 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:31:22,943 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:31:23,207 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:31:23,207 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 14:31:23,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 14:31:23.465 UTC [4608] WARNING: there is no transaction in progress
2026-08-11 14:31:23,474 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 14:31:23,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:31:23,735 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 14:31:23,736 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:31:23,736 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 14:31:23,988 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 14:31:24,239 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbsf.c.gcp-upstream.internal','N') ;"
2026-08-11 14:31:24,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:31:24,761 [I] Table x_portal_user does not exist in database ranger
2026-08-11 14:31:24,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:31:25,049 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 14:31:25,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:31:25,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:31:25,556 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 14:31:25,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 14:31:26.594 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.611 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.627 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.643 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.661 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.678 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.733 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.741 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.756 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.766 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.777 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.791 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.801 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.808 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.815 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:26.822 UTC [4732] WARNING: there is no transaction in progress
2026-08-11 14:31:27,502 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 14:31:27,502 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 14:31:27,766 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 14:31:27,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:31:28,043 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 14:31:28,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:31:28,314 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 14:31:28,314 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:31:28,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:31:28,831 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbsf.c.gcp-upstream.internal';"
2026-08-11 14:31:29,096 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 14:31:29,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 14:31:29,354 [I] DB_PATCHES have already been applied
I20260811 14:31:29.360205 24299 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLI13IK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 14:31:29.360282 24299 mini_ranger.cc:193] Using host: 127.23.186.212
I20260811 14:31:29.365180 24299 mini_ranger.cc:227] Ranger admin URL: http://127.23.186.212:37583
Aug 11, 2026 2:31:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:31:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:31:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:31:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:31:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:31:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:31:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 14:31:30.573362 24299 mini_ranger.cc:160] Time spent starting Ranger: real 9.014s user 0.000s sys 0.012s
Aug 11, 2026 2:31:52 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:31:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 2:31:52 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:31:52 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:31:53 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 14:31:53.589591 24299 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/rangerkms_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/rangerkms_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.186.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.186.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 14:31:53 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458713, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 14:31:53.684091 24299 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 14:31:53.695989 24299 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 14:31:54.606022 24299 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 14:31:54.606117 24299 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms
I20260811 14:31:54.606141 24299 mini_ranger_kms.cc:227] Using postgres at 127.23.186.212:40777
2026-08-11 14:31:54,856 [I] DB FLAVOR :POSTGRES
2026-08-11 14:31:54,856 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:31:54,856 [I] Checking connection
2026-08-11 14:31:54,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 14:31:55,137 [I] connection success
2026-08-11 14:31:55,137 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 14:31:55,138 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 14:31:55,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 14:31:55,401 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 14:31:55,401 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:31:55,401 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 14:31:55,402 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:40777/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 14:31:55,664 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 14:31:56.204409 24299 mini_ranger_kms.cc:310] Created kms service
I20260811 14:31:56.428563 24299 mini_ranger_kms.cc:326] Created kudu user
I20260811 14:31:56.479921 24299 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 14:31:56 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458713, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:31:56.888726 24299 mini_ranger_kms.cc:384] Added ranger policy
I20260811 14:31:56.888860 24299 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 14:31:56.888882 24299 mini_ranger_kms.cc:240] Using host: 127.23.186.212
I20260811 14:31:56.893064 24299 mini_ranger_kms.cc:276] Ranger KMS PID: 4955
I20260811 14:31:56.893200 24299 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.186.212:54357
Aug 11, 2026 2:31:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:31:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:31:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:31:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:31:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:31:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 14:31:59.415788 24299 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.732s user 0.013s sys 0.219s
I20260811 14:31:59.415906 24299 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 14:31:59.415992 24299 mini_ranger_kms.cc:401] 127.23.186.212:54357/kms/v1/keys
Aug 11 14:31:59 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786458719, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.186.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 14:31:59 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458719, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.186.212@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
Aug 11 14:32:00 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458719, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.186.212@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
Aug 11 14:32:02 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:32:02.768360 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:46071
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:46071
--ranger_config_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54357/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:32:02.857658 5016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:32:02.857899 5016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:32:02.857924 5016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:32:02.859222 5016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:32:02.859284 5016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:32:02.859299 5016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:32:02.859313 5016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:32:02.859325 5016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:32:02.861202 5016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.186.212:54357/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:46071
--ranger_config_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:46071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:32:02.861498 5016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:32:02.861773 5016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:32:02.867622 5024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:02.867651 5022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:32:02.867746 5016 server_base.cc:1034] running on GCE node
W20260811 14:32:02.867744 5021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:32:02.868214 5016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:32:02.868559 5016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:32:02.869750 5016 hybrid_clock.cc:648] HybridClock initialized: now 1786458722869733 us; error 54 us; skew 500 ppm
Aug 11 14:32:02 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:32:02.877393 5016 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:32:02.877992 5016 webserver.cc:460] Webserver started at http://127.23.186.254:39417/ using document root <none> and password file <none>
I20260811 14:32:02.878238 5016 fs_manager.cc:359] Metadata directory not provided
I20260811 14:32:02.878302 5016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:32:02.878472 5016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:32:02 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458722, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:32:02.936609 5016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "feb6a10d51d84ec58b41a303cf78e2fe"
format_stamp: "Formatted at 2026-08-11 14:32:02 on dist-test-slave-cbsf"
server_key: "3dcf2daab82c22115664008a0eff298c"
server_key_iv: "373f9866080e40eba1098c9901ea8d62"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:02.937188 5016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "feb6a10d51d84ec58b41a303cf78e2fe"
format_stamp: "Formatted at 2026-08-11 14:32:02 on dist-test-slave-cbsf"
server_key: "3dcf2daab82c22115664008a0eff298c"
server_key_iv: "373f9866080e40eba1098c9901ea8d62"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:02.939251 5016 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 14:32:02.955705 5032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:02.955988 5016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:32:02.956125 5016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "feb6a10d51d84ec58b41a303cf78e2fe"
format_stamp: "Formatted at 2026-08-11 14:32:02 on dist-test-slave-cbsf"
server_key: "3dcf2daab82c22115664008a0eff298c"
server_key_iv: "373f9866080e40eba1098c9901ea8d62"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:02.956231 5016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:32:02.986366 5016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:32:02.988461 5016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:32:02.988656 5016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:32:02.994632 5016 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:46071
I20260811 14:32:02.994706 5093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:46071 every 8 connection(s)
I20260811 14:32:02.995321 5016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:32:02.996381 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 5016
I20260811 14:32:02.996506 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:32:02.996768 24299 external_mini_cluster.cc:1428] Setting key 17e507809206083b7c4e2aa024d503a6
I20260811 14:32:02.998291 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:32:03.001907 5094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe: Bootstrap starting.
I20260811 14:32:03.002698 5094 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe: Neither blocks nor log segments found. Creating new log.
I20260811 14:32:03.003089 5094 log.cc:824] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe: Log is configured to *not* fsync() on all Append() calls
I20260811 14:32:03.004007 5094 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe: No bootstrap required, opened a new log
I20260811 14:32:03.006151 5094 raft_consensus.cc:348] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46071 } }
I20260811 14:32:03.006352 5094 raft_consensus.cc:374] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:32:03.006389 5094 raft_consensus.cc:729] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb6a10d51d84ec58b41a303cf78e2fe, State: Initialized, Role: FOLLOWER
I20260811 14:32:03.006577 5094 consensus_queue.cc:260] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46071 } }
I20260811 14:32:03.006675 5094 raft_consensus.cc:388] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:32:03.006708 5094 raft_consensus.cc:482] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:32:03.006778 5094 raft_consensus.cc:3037] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:32:03.007615 5094 raft_consensus.cc:504] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46071 } }
I20260811 14:32:03.007786 5094 leader_election.cc:302] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: feb6a10d51d84ec58b41a303cf78e2fe; no voters:
I20260811 14:32:03.008039 5094 leader_election.cc:288] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:32:03.008073 5102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:32:03.008303 5102 raft_consensus.cc:686] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [term 1 LEADER]: Becoming Leader. State: Replica: feb6a10d51d84ec58b41a303cf78e2fe, State: Running, Role: LEADER
I20260811 14:32:03.008373 5094 sys_catalog.cc:564] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:32:03.008504 5102 consensus_queue.cc:237] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46071 } }
I20260811 14:32:03.009006 5102 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46071 } } }
I20260811 14:32:03.009186 5102 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [sys.catalog]: This master's current role is: LEADER
I20260811 14:32:03.009390 5102 sys_catalog.cc:455] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader feb6a10d51d84ec58b41a303cf78e2fe. Latest consensus state: current_term: 1 leader_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "feb6a10d51d84ec58b41a303cf78e2fe" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46071 } } }
I20260811 14:32:03.009505 5102 sys_catalog.cc:458] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe [sys.catalog]: This master's current role is: LEADER
I20260811 14:32:03.009687 5112 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:32:03.010355 5112 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:32:03.010897 5094 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 11 14:32:03 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458722, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:32:03.013335 5112 catalog_manager.cc:1347] Generated new cluster ID: f769157bf35147d79d5be31a75ea3e41
I20260811 14:32:03.013447 5112 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:32:03.017324 5100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:02.997485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.23.186.254:46071)
0811 14:32:02.997701 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:02.997713 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:03.003651 (+ 5938us) server_negotiation.cc:407] Connection header received
0811 14:32:03.004495 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:03.004506 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:03.004582 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:03.004782 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:32:03.005611 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:03.007622 (+ 2011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:03.008845 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:03.009545 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:03.009635 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:03.012924 (+ 3289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:32:03.012928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:32:03.012929 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:32:03.012955 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:32:03.015298 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:03.015773 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:03.015777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:03.015778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:03.015885 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:03.016105 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:03.016106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:03.016106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:03.016436 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:32:03.016603 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:03.016916 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 14:32:03.017074 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260811 14:32:03.045344 5112 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:32:03.046293 5112 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:32:03.052714 5112 catalog_manager.cc:5868] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe: Generated new TSK 0
I20260811 14:32:03.053041 5112 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:32:03.866398 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46071
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:32:03.975982 5153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:32:03.976207 5153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:32:03.976233 5153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:32:03.977537 5153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:32:03.977586 5153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:32:03.977615 5153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:32:03.979624 5153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.186.212:54357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:32:03.979913 5153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:32:03.980175 5153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:32:03.985960 5160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:03.986079 5159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:03.985991 5162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:32:03.986408 5153 server_base.cc:1034] running on GCE node
I20260811 14:32:03.986624 5153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:32:03.986910 5153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:32:03.990175 5153 hybrid_clock.cc:648] HybridClock initialized: now 1786458723989128 us; error 1057 us; skew 500 ppm
Aug 11 14:32:03 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:32:03.993618 5153 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:32:03.994179 5153 webserver.cc:460] Webserver started at http://127.23.186.193:46297/ using document root <none> and password file <none>
I20260811 14:32:03.994429 5153 fs_manager.cc:359] Metadata directory not provided
I20260811 14:32:03.994513 5153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:32:03.994656 5153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:32:03 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:32:04.015931 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "0e1828e4db124609be6c9309dcc3c9f0"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "c535261c446be7818f6c8439563ba7f7"
server_key_iv: "b951d63c988aeb013c38f1325ec77fb6"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.016407 5153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "0e1828e4db124609be6c9309dcc3c9f0"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "c535261c446be7818f6c8439563ba7f7"
server_key_iv: "b951d63c988aeb013c38f1325ec77fb6"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.018442 5153 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 14:32:04.038322 5172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.038872 5153 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260811 14:32:04.039167 5153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "0e1828e4db124609be6c9309dcc3c9f0"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "c535261c446be7818f6c8439563ba7f7"
server_key_iv: "b951d63c988aeb013c38f1325ec77fb6"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.039335 5153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:32:04.088253 5153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:32:04.090736 5153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:32:04.090920 5153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:32:04.091253 5153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:32:04.091714 5153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:32:04.091771 5153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.091816 5153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:32:04.091843 5153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.099942 5153 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:40155
I20260811 14:32:04.100420 5153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:32:04.105598 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 5153
I20260811 14:32:04.105736 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:32:04.105940 24299 external_mini_cluster.cc:1428] Setting key ef1f0c366e41cdaba546ae137c118ddd
I20260811 14:32:04.113826 5313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:40155 every 8 connection(s)
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458723, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:32:04.134266 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.113475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:40367 (local address 127.23.186.254:46071)
0811 14:32:04.120827 (+ 7352us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.120830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.123426 (+ 2596us) server_negotiation.cc:407] Connection header received
0811 14:32:04.123474 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.123475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.123496 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.124359 (+ 863us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:32:04.124510 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.125377 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.126626 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.126946 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.127017 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.129944 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:32:04.129945 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:32:04.129946 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:32:04.129956 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:32:04.132448 (+ 2492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.133029 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.133030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.133031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.133099 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.133425 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.133427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.133427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.133602 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:32:04.133659 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.134053 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.134153 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7318,"thread_start_us":71,"threads_started":1}
I20260811 14:32:04.134856 5318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.113486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:46071 (local address 127.23.186.193:40367)
0811 14:32:04.120834 (+ 7348us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.120914 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.121672 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.123728 (+ 2056us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.123735 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.124045 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:32:04.124385 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.124393 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.125501 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.125502 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.126512 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.126515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.127101 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.127102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.127261 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.127927 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:32:04.127939 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:32:04.129813 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:32:04.132570 (+ 2757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:32:04.132578 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:32:04.132581 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:32:04.132922 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:32:04.133184 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:32:04.133185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:32:04.133186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:32:04.133333 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:32:04.133701 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:32:04.133709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:32:04.133884 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.134398 (+ 514us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.134587 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7271,"thread_start_us":101,"threads_started":1}
I20260811 14:32:04.135526 5314 heartbeater.cc:347] Connected to a master server at 127.23.186.254:46071
I20260811 14:32:04.135658 5314 heartbeater.cc:464] Registering TS with master...
I20260811 14:32:04.135893 5314 heartbeater.cc:511] Master 127.23.186.254:46071 requested a full tablet report, sending...
I20260811 14:32:04.136576 5055 ts_manager.cc:194] Registered new tserver with Master: 0e1828e4db124609be6c9309dcc3c9f0 (127.23.186.193:40155)
I20260811 14:32:04.137259 5055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:40367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:32:04.144394 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46071
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 14:32:04.247010 5321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:32:04.247287 5321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:32:04.247332 5321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:32:04.248646 5321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:32:04.248826 5321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:32:04.248919 5321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:32:04.250886 5321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.186.212:54357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:32:04.251194 5321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:32:04.251478 5321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:32:04.257653 5330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:04.257977 5328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:04.258097 5327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:32:04.258534 5321 server_base.cc:1034] running on GCE node
I20260811 14:32:04.258766 5321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:32:04.259117 5321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:32:04.260284 5321 hybrid_clock.cc:648] HybridClock initialized: now 1786458724260273 us; error 30 us; skew 500 ppm
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:32:04.263672 5321 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:32:04.264245 5321 webserver.cc:460] Webserver started at http://127.23.186.194:44197/ using document root <none> and password file <none>
I20260811 14:32:04.264573 5321 fs_manager.cc:359] Metadata directory not provided
I20260811 14:32:04.264650 5321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:32:04.264830 5321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:32:04.289867 5321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "07713f07c9884753a1a1bcbaf0cc1cf7"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "b238d4ff0639332ee4daafdc579a585d"
server_key_iv: "69f44ab7e9b87a2a9e906511c6fcded3"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.290737 5321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "07713f07c9884753a1a1bcbaf0cc1cf7"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "b238d4ff0639332ee4daafdc579a585d"
server_key_iv: "69f44ab7e9b87a2a9e906511c6fcded3"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.294924 5321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.001s
I20260811 14:32:04.317306 5340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.317508 5321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 14:32:04.317617 5321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "07713f07c9884753a1a1bcbaf0cc1cf7"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "b238d4ff0639332ee4daafdc579a585d"
server_key_iv: "69f44ab7e9b87a2a9e906511c6fcded3"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.317747 5321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:32:04.345916 5321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:32:04.348654 5321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:32:04.348846 5321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:32:04.349110 5321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:32:04.349555 5321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:32:04.349620 5321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.349663 5321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:32:04.349695 5321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.359877 5321 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33379
I20260811 14:32:04.360489 5321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:32:04.362048 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 5321
I20260811 14:32:04.362167 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:32:04.362370 24299 external_mini_cluster.cc:1428] Setting key 9812fed52c131904cef085f67db07277
I20260811 14:32:04.360947 5482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33379 every 8 connection(s)
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:32:04.373982 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.362089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:50903 (local address 127.23.186.254:46071)
0811 14:32:04.362145 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.362147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.362584 (+ 437us) server_negotiation.cc:407] Connection header received
0811 14:32:04.363411 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.363412 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.363430 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.363464 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:32:04.364369 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.365193 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.366931 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.367240 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.367294 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.370301 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:32:04.370303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:32:04.370304 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:32:04.370313 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:32:04.372146 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.372742 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.372744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.372745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.372815 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.373165 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.373166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.373167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.373322 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:32:04.373366 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.373771 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.373881 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:32:04.374650 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.361757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:46071 (local address 127.23.186.194:50903)
0811 14:32:04.362433 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.362482 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.363218 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.363667 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.363671 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.363932 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:32:04.364249 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.364255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.365514 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.365515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.366824 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.366827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.367400 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.367400 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.367551 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.368269 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:32:04.368280 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:32:04.370172 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:32:04.372257 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:32:04.372266 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:32:04.372269 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:32:04.372607 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:32:04.372899 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:32:04.372899 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:32:04.372900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:32:04.373068 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:32:04.373415 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:32:04.373423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:32:04.373604 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.374173 (+ 569us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.374369 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":105,"threads_started":1}
I20260811 14:32:04.375231 5483 heartbeater.cc:347] Connected to a master server at 127.23.186.254:46071
I20260811 14:32:04.375340 5483 heartbeater.cc:464] Registering TS with master...
I20260811 14:32:04.375554 5483 heartbeater.cc:511] Master 127.23.186.254:46071 requested a full tablet report, sending...
I20260811 14:32:04.376029 5055 ts_manager.cc:194] Registered new tserver with Master: 07713f07c9884753a1a1bcbaf0cc1cf7 (127.23.186.194:33379)
I20260811 14:32:04.376540 5055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:50903
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:32:04.423880 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54357/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46071
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:32947
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:32:04.548794 5488 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:32:04.549171 5488 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:32:04.549256 5488 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:32:04.551016 5488 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:32:04.551141 5488 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:32:04.551224 5488 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:32:04.553462 5488 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:32947
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.186.212:54357/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:46071
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5488
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:32:04.554008 5488 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:32:04.554333 5488 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:32:04.562031 5494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:04.562420 5495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:32:04.562661 5498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:32:04.563540 5488 server_base.cc:1034] running on GCE node
I20260811 14:32:04.563789 5488 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:32:04.564116 5488 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:32:04.565316 5488 hybrid_clock.cc:648] HybridClock initialized: now 1786458724565300 us; error 29 us; skew 500 ppm
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:32:04.571561 5488 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:32:04.572225 5488 webserver.cc:460] Webserver started at http://127.23.186.195:40781/ using document root <none> and password file <none>
I20260811 14:32:04.572522 5488 fs_manager.cc:359] Metadata directory not provided
I20260811 14:32:04.572593 5488 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:32:04.572791 5488 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:32:04.592980 5488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "b50e48e05a2941359a3a69126795987c"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "6d43004c741717f42a9b09013ec88eec"
server_key_iv: "8562f284ff7a747a2383a41ce2b1cac5"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.593637 5488 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "b50e48e05a2941359a3a69126795987c"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "6d43004c741717f42a9b09013ec88eec"
server_key_iv: "8562f284ff7a747a2383a41ce2b1cac5"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.595638 5488 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 14:32:04.621629 5505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.622637 5488 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 14:32:04.622781 5488 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "b50e48e05a2941359a3a69126795987c"
format_stamp: "Formatted at 2026-08-11 14:32:04 on dist-test-slave-cbsf"
server_key: "6d43004c741717f42a9b09013ec88eec"
server_key_iv: "8562f284ff7a747a2383a41ce2b1cac5"
server_key_version: "kuduclusterkey@0"
I20260811 14:32:04.622932 5488 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:32:04.637544 5488 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:32:04.640081 5488 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:32:04.640254 5488 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:32:04.640532 5488 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:32:04.640990 5488 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:32:04.641045 5488 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.641088 5488 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:32:04.641111 5488 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:32:04.651538 5488 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:42863
I20260811 14:32:04.652307 5488 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:32:04.652966 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 5488
I20260811 14:32:04.653095 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:32:04.653302 24299 external_mini_cluster.cc:1428] Setting key 47692a665e3d3dde00b1232b14e2a4c6
I20260811 14:32:04.654448 5634 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:42863 every 8 connection(s)
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:32:04.668948 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.655671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:46155 (local address 127.23.186.254:46071)
0811 14:32:04.655722 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.655725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.656969 (+ 1244us) server_negotiation.cc:407] Connection header received
0811 14:32:04.657799 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.657800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.657813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.657845 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:32:04.658735 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.659510 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.661143 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.661444 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.661496 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.664746 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:32:04.664747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:32:04.664747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:32:04.664755 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:32:04.667178 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.667728 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.667730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.667730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.667790 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.668068 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.668069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.668070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.668218 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:32:04.668263 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.668736 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.668832 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:32:04.669814 5637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.655347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:46071 (local address 127.23.186.195:46155)
0811 14:32:04.656829 (+ 1482us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.656881 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.657669 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.658039 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.658043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.658309 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:32:04.658623 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.658629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.659632 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.659633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.661037 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.661040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.661614 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.661615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.661764 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.662476 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:32:04.662500 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:32:04.664305 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:32:04.667288 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:32:04.667297 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:32:04.667300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:32:04.667640 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:32:04.667859 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:32:04.667859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:32:04.667860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:32:04.667990 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:32:04.668312 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:32:04.668319 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:32:04.668518 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.669354 (+ 836us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.669558 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1428,"thread_start_us":124,"threads_started":1}
I20260811 14:32:04.670416 5635 heartbeater.cc:347] Connected to a master server at 127.23.186.254:46071
I20260811 14:32:04.670521 5635 heartbeater.cc:464] Registering TS with master...
I20260811 14:32:04.670745 5635 heartbeater.cc:511] Master 127.23.186.254:46071 requested a full tablet report, sending...
I20260811 14:32:04.671190 5055 ts_manager.cc:194] Registered new tserver with Master: b50e48e05a2941359a3a69126795987c (127.23.186.195:42863)
I20260811 14:32:04.671712 5055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:46155
I20260811 14:32:04.676571 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:32:04.687335 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.678716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.23.186.254:46071)
0811 14:32:04.678934 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.678936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.678943 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:32:04.678967 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.678968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.678978 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.679022 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:32:04.679836 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.680623 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.681792 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.682096 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.682140 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.683828 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:32:04.683830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:32:04.683830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:32:04.683839 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:32:04.685977 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.686429 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.686430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.686430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.686500 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:32:04.686725 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:32:04.686726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:32:04.686726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:32:04.686878 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:32:04.686983 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.687137 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.687229 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194}
I20260811 14:32:04.689539 5055 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:32:04.692308 5055 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:32:04.701020 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.695863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.23.186.193:40155)
0811 14:32:04.696002 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.696008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.696334 (+ 326us) server_negotiation.cc:407] Connection header received
0811 14:32:04.696481 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.696484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.696501 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.696566 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:32:04.696885 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.698058 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.699625 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.700399 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.700452 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.700512 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.700873 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.700922 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":75,"threads_started":1}
I20260811 14:32:04.701301 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.695765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:40155 (local address 127.0.0.1:45058)
0811 14:32:04.696206 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.696252 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.696377 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.696678 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.696682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.696684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:32:04.696796 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.696798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.698163 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.698164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.699510 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.699512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.700589 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.700590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.700736 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.700754 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.701149 (+ 395us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.701225 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":86,"threads_started":1}
I20260811 14:32:04.702437 5218 tablet_service.cc:1467] Processing CreateTablet for tablet 47a92825b0664011bd71887b2aba3e36 (DEFAULT_TABLE table=test-table [id=2e86807e5ddd4057bce349654e128054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:32:04.702857 5218 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a92825b0664011bd71887b2aba3e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:32:04.705631 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.696208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:42863 (local address 127.0.0.1:48186)
0811 14:32:04.701376 (+ 5168us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.701387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.701435 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.701749 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.701750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.701752 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:32:04.701782 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.701784 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.703086 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.703087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.704212 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.704214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.705373 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.705374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.705438 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.705442 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.705492 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.705551 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5156,"thread_start_us":63,"threads_started":1}
I20260811 14:32:04.705940 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.696756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.23.186.195:42863)
0811 14:32:04.697429 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.697432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.701500 (+ 4068us) server_negotiation.cc:407] Connection header received
0811 14:32:04.701569 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.701572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.701590 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.701664 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:32:04.701849 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.702984 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.704346 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.705186 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.705276 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.705335 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.705805 (+ 470us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.705853 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":76,"threads_started":1}
I20260811 14:32:04.706622 5562 tablet_service.cc:1467] Processing CreateTablet for tablet 47a92825b0664011bd71887b2aba3e36 (DEFAULT_TABLE table=test-table [id=2e86807e5ddd4057bce349654e128054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:32:04.706980 5562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a92825b0664011bd71887b2aba3e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:32:04.709249 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.696030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33379 (local address 127.0.0.1:56480)
0811 14:32:04.697488 (+ 1458us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.697499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.697557 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.697834 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.697835 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.697837 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:32:04.697893 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.697895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.699556 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.699557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.700778 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.700782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.708981 (+ 8199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.708983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.709065 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.709070 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.709117 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.709168 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1437,"thread_start_us":95,"threads_started":1}
I20260811 14:32:04.709911 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.696549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56480 (local address 127.23.186.194:33379)
0811 14:32:04.697323 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.697328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.697613 (+ 285us) server_negotiation.cc:407] Connection header received
0811 14:32:04.697660 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.697663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.697679 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.697752 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:32:04.697977 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.699457 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.700905 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.708675 (+ 7770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.708847 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.709734 (+ 887us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.709768 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.709820 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":732,"thread_start_us":83,"threads_started":1}
I20260811 14:32:04.710948 5655 tablet_bootstrap.cc:492] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0: Bootstrap starting.
I20260811 14:32:04.710875 5406 tablet_service.cc:1467] Processing CreateTablet for tablet 47a92825b0664011bd71887b2aba3e36 (DEFAULT_TABLE table=test-table [id=2e86807e5ddd4057bce349654e128054]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:32:04.711272 5406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 47a92825b0664011bd71887b2aba3e36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:32:04.711755 5655 tablet_bootstrap.cc:654] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0: Neither blocks nor log segments found. Creating new log.
I20260811 14:32:04.712145 5655 log.cc:824] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:32:04.712531 5657 tablet_bootstrap.cc:492] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Bootstrap starting.
I20260811 14:32:04.713656 5657 tablet_bootstrap.cc:654] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Neither blocks nor log segments found. Creating new log.
I20260811 14:32:04.713990 5657 log.cc:824] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Log is configured to *not* fsync() on all Append() calls
I20260811 14:32:04.714968 5660 tablet_bootstrap.cc:492] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: Bootstrap starting.
I20260811 14:32:04.715989 5660 tablet_bootstrap.cc:654] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: Neither blocks nor log segments found. Creating new log.
I20260811 14:32:04.716332 5660 log.cc:824] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: Log is configured to *not* fsync() on all Append() calls
I20260811 14:32:04.717242 5660 tablet_bootstrap.cc:492] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: No bootstrap required, opened a new log
I20260811 14:32:04.717312 5660 ts_tablet_manager.cc:1397] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:32:04.719362 5660 raft_consensus.cc:348] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.719568 5660 raft_consensus.cc:374] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:32:04.719602 5660 raft_consensus.cc:729] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07713f07c9884753a1a1bcbaf0cc1cf7, State: Initialized, Role: FOLLOWER
I20260811 14:32:04.719756 5660 consensus_queue.cc:260] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.720013 5660 ts_tablet_manager.cc:1428] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:32:04.720130 5483 heartbeater.cc:503] Master 127.23.186.254:46071 was elected leader, sending a full tablet report...
I20260811 14:32:04.721508 5655 tablet_bootstrap.cc:492] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0: No bootstrap required, opened a new log
I20260811 14:32:04.721592 5655 ts_tablet_manager.cc:1397] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260811 14:32:04.724514 5655 raft_consensus.cc:348] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.724722 5655 raft_consensus.cc:374] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:32:04.724776 5655 raft_consensus.cc:729] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1828e4db124609be6c9309dcc3c9f0, State: Initialized, Role: FOLLOWER
I20260811 14:32:04.724908 5655 consensus_queue.cc:260] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.725225 5655 ts_tablet_manager.cc:1428] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260811 14:32:04.725391 5314 heartbeater.cc:503] Master 127.23.186.254:46071 was elected leader, sending a full tablet report...
I20260811 14:32:04.726949 5657 tablet_bootstrap.cc:492] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: No bootstrap required, opened a new log
I20260811 14:32:04.727053 5657 ts_tablet_manager.cc:1397] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.000s
I20260811 14:32:04.729439 5657 raft_consensus.cc:348] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.729635 5657 raft_consensus.cc:374] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:32:04.729683 5657 raft_consensus.cc:729] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b50e48e05a2941359a3a69126795987c, State: Initialized, Role: FOLLOWER
I20260811 14:32:04.729799 5657 consensus_queue.cc:260] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.730144 5657 ts_tablet_manager.cc:1428] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:32:04.730190 5635 heartbeater.cc:503] Master 127.23.186.254:46071 was elected leader, sending a full tablet report...
Aug 11 14:32:04 dist-test-slave-cbsf krb5kdc[4485](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458724, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:32:04.756255 5662 raft_consensus.cc:482] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:32:04.756423 5662 raft_consensus.cc:504] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.757102 5662 leader_election.cc:288] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0e1828e4db124609be6c9309dcc3c9f0 (127.23.186.193:40155), b50e48e05a2941359a3a69126795987c (127.23.186.195:42863)
I20260811 14:32:04.761324 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.757404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:42863 (local address 127.23.186.194:59497)
0811 14:32:04.757486 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.757501 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.757563 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.757864 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.757865 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.757866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:32:04.757965 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.757966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.758973 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.758975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.760170 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.760172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.761053 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.761054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.761145 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.761149 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.761205 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.761247 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 14:32:04.761402 5654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.757672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:59497 (local address 127.23.186.195:42863)
0811 14:32:04.757720 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.757722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.757729 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:32:04.757759 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.757760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.757768 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.757813 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:32:04.758044 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.758830 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.760276 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.760899 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.760969 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.760979 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.761245 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.761316 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:32:04.761947 5580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a92825b0664011bd71887b2aba3e36" candidate_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e48e05a2941359a3a69126795987c" is_pre_election: true
I20260811 14:32:04.762145 5580 raft_consensus.cc:2454] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07713f07c9884753a1a1bcbaf0cc1cf7 in term 0.
I20260811 14:32:04.763643 5344 leader_election.cc:302] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07713f07c9884753a1a1bcbaf0cc1cf7, b50e48e05a2941359a3a69126795987c; no voters:
I20260811 14:32:04.763787 5662 raft_consensus.cc:2781] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:32:04.763825 5662 raft_consensus.cc:482] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:32:04.763846 5662 raft_consensus.cc:3037] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:32:04.764966 5662 raft_consensus.cc:504] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.765146 5662 leader_election.cc:288] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [CANDIDATE]: Term 1 election: Requested vote from peers 0e1828e4db124609be6c9309dcc3c9f0 (127.23.186.193:40155), b50e48e05a2941359a3a69126795987c (127.23.186.195:42863)
I20260811 14:32:04.765290 5649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.757838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:44227 (local address 127.23.186.193:40155)
0811 14:32:04.757884 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.757886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.761737 (+ 3851us) server_negotiation.cc:407] Connection header received
0811 14:32:04.761772 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.761773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.761784 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.761830 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:32:04.762294 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.763015 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.764170 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.764842 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.764915 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.764925 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.765178 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.765224 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:32:04.765668 5580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a92825b0664011bd71887b2aba3e36" candidate_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b50e48e05a2941359a3a69126795987c"
I20260811 14:32:04.765775 5580 raft_consensus.cc:3037] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:32:04.766708 5580 raft_consensus.cc:2454] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07713f07c9884753a1a1bcbaf0cc1cf7 in term 1.
I20260811 14:32:04.766914 5344 leader_election.cc:302] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07713f07c9884753a1a1bcbaf0cc1cf7, b50e48e05a2941359a3a69126795987c; no voters:
I20260811 14:32:04.768998 5662 raft_consensus.cc:2781] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:32:04.769218 5662 raft_consensus.cc:686] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 1 LEADER]: Becoming Leader. State: Replica: 07713f07c9884753a1a1bcbaf0cc1cf7, State: Running, Role: LEADER
I20260811 14:32:04.769359 5662 consensus_queue.cc:237] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } }
I20260811 14:32:04.770380 5055 catalog_manager.cc:5495] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 07713f07c9884753a1a1bcbaf0cc1cf7 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33379 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 } health_report { overall_health: UNKNOWN } } }
I20260811 14:32:04.773561 5485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.757543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:40155 (local address 127.23.186.194:44227)
0811 14:32:04.761421 (+ 3878us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.761432 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:32:04.761476 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:32:04.761873 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:32:04.761874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:32:04.761875 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:32:04.761904 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.761905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.763088 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.763090 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.764088 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:32:04.764090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.764959 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.764960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:32:04.765024 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.765028 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:32:04.765074 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 14:32:04.765112 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3856,"thread_start_us":153,"threads_started":1}
I20260811 14:32:04.774184 5260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a92825b0664011bd71887b2aba3e36" candidate_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1828e4db124609be6c9309dcc3c9f0" is_pre_election: true
I20260811 14:32:04.774405 5260 raft_consensus.cc:2454] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07713f07c9884753a1a1bcbaf0cc1cf7 in term 0.
I20260811 14:32:04.774578 5259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "47a92825b0664011bd71887b2aba3e36" candidate_uuid: "07713f07c9884753a1a1bcbaf0cc1cf7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1828e4db124609be6c9309dcc3c9f0"
I20260811 14:32:04.774667 5259 raft_consensus.cc:3037] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:32:04.775722 5259 raft_consensus.cc:2454] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07713f07c9884753a1a1bcbaf0cc1cf7 in term 1.
I20260811 14:32:04.785282 5653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.781351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.23.186.194:33379)
0811 14:32:04.781393 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:32:04.781395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:32:04.781402 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:32:04.781453 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:32:04.781454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:32:04.781475 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:32:04.781539 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:32:04.782032 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.782790 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.783942 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:32:04.784267 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:32:04.784348 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:32:04.784597 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:32:04.784823 (+ 226us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:32:04.784924 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 14:32:04.785134 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 14:32:04.785160 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:32:04.789044 5259 raft_consensus.cc:1261] T 47a92825b0664011bd71887b2aba3e36 P 0e1828e4db124609be6c9309dcc3c9f0 [term 1 FOLLOWER]: Refusing update from remote peer 07713f07c9884753a1a1bcbaf0cc1cf7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:32:04.789094 5580 raft_consensus.cc:1261] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 1 FOLLOWER]: Refusing update from remote peer 07713f07c9884753a1a1bcbaf0cc1cf7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:32:04.789557 5662 consensus_queue.cc:1035] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0e1828e4db124609be6c9309dcc3c9f0" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 40155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:32:04.789726 5662 consensus_queue.cc:1035] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b50e48e05a2941359a3a69126795987c" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 42863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:32:04.793299 5671 mvcc.cc:205] Tried to move back new op lower bound from 7317334936732286976 to 7317334936656031744. Current Snapshot: MvccSnapshot[applied={T|T < 7317334936732286976}]
I20260811 14:32:04.795823 5674 mvcc.cc:205] Tried to move back new op lower bound from 7317334936732286976 to 7317334936656031744. Current Snapshot: MvccSnapshot[applied={T|T < 7317334936732286976}]
I20260811 14:32:04.798691 5054 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:32:04.798804 5054 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:32:04.799726 5054 catalog_manager.cc:5782] T 00000000000000000000000000000000 P feb6a10d51d84ec58b41a303cf78e2fe: Sending DeleteTablet for 3 replicas of tablet 47a92825b0664011bd71887b2aba3e36
I20260811 14:32:04.800105 5218 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a92825b0664011bd71887b2aba3e36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:32:04 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:45058
I20260811 14:32:04.800649 5406 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a92825b0664011bd71887b2aba3e36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:32:04 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:56480
I20260811 14:32:04.800956 5562 tablet_service.cc:1514] Processing DeleteTablet for tablet 47a92825b0664011bd71887b2aba3e36 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:32:04 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:48186
I20260811 14:32:04.801656 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 5153
I20260811 14:32:04.808168 5683 tablet_replica.cc:330] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7: stopping tablet replica
I20260811 14:32:04.808321 5683 raft_consensus.cc:2229] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:32:04.808449 5683 raft_consensus.cc:2258] T 47a92825b0664011bd71887b2aba3e36 P 07713f07c9884753a1a1bcbaf0cc1cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:32:04.808619 5684 tablet_replica.cc:330] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: stopping tablet replica
I20260811 14:32:04.809358 5684 raft_consensus.cc:2229] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:32:04.809509 5684 raft_consensus.cc:2258] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:32:04.810747 5684 ts_tablet_manager.cc:1905] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:32:04.812494 5684 ts_tablet_manager.cc:1918] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:32:04.812564 5684 log.cc:1196] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/47a92825b0664011bd71887b2aba3e36
W20260811 14:32:04.819864 5345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:40155 (error 108)
I20260811 14:32:04.820825 5684 ts_tablet_manager.cc:1939] T 47a92825b0664011bd71887b2aba3e36 P b50e48e05a2941359a3a69126795987c: Deleting consensus metadata
I20260811 14:32:04.822661 5035 catalog_manager.cc:4841] TS b50e48e05a2941359a3a69126795987c (127.23.186.195:42863): tablet 47a92825b0664011bd71887b2aba3e36 (table test-table [id=2e86807e5ddd4057bce349654e128054]) successfully deleted
W20260811 14:32:04.827778 5036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:40155 (error 108)
I20260811 14:32:04.829118 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 5321
I20260811 14:32:04.829130 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.828955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:40155 (local address 127.0.0.1:45068)
0811 14:32:04.829016 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.829089 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 14:32:04.829200 5667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.828967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:40155 (local address 127.23.186.194:45615)
0811 14:32:04.829077 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.829138 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:40155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:32:04.862175 5036 catalog_manager.cc:4584] TS 0e1828e4db124609be6c9309dcc3c9f0 (127.23.186.193:40155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 47a92825b0664011bd71887b2aba3e36: Network error: Client connection negotiation failed: client connection to 127.23.186.193:40155: connect: Connection refused (error 111)
I20260811 14:32:04.868968 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 5488
I20260811 14:32:04.869221 5651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:32:04.868950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33379 (local address 127.0.0.1:56500)
0811 14:32:04.869141 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:32:04.869176 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.194:33379: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 14:32:04.880008 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 5016
I20260811 14:32:04.890318 24299 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 14:32:05.235611 24299 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 14:32:05.235721 24299 mini_ranger.cc:66] Stopping Ranger...
I20260811 14:32:05.591641 24299 mini_ranger.cc:68] Stopped Ranger
2026-08-11 14:32:05.591 UTC [4520] LOG: received smart shutdown request
2026-08-11 14:32:05.596 UTC [4520] LOG: background worker "logical replication launcher" (PID 4529) exited with exit code 1
2026-08-11 14:32:05.596 UTC [4524] LOG: shutting down
2026-08-11 14:32:05.762 UTC [4520] LOG: database system is shut down
2026-08-11T14:32:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49343 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:32:05 dist-test-slave-cbsf krb5kdc[5698](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:32:05 dist-test-slave-cbsf krb5kdc[5698](info): set up 2 sockets
Aug 11 14:32:05 dist-test-slave-cbsf krb5kdc[5698](info): commencing operation
krb5kdc: starting...
W20260811 14:32:06.998351 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:32:07 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:32:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:32:07Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/rangeradmin_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/rangeradmin_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/rangerlookup_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/rangerlookup_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
I20260811 14:32:07.189270 24299 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLI13IK/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/postgres -l logfile start
2026-08-11 14:32:09.687 UTC [5737] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 14:32:09.687 UTC [5737] LOG: listening on IPv4 address "127.23.186.212", port 50215
2026-08-11 14:32:09.691 UTC [5737] LOG: listening on Unix socket "/tmp/.s.PGSQL.50215"
2026-08-11 14:32:09.727 UTC [5740] LOG: database system was shut down at 2026-08-11 14:32:08 UTC
2026-08-11 14:32:09.731 UTC [5737] LOG: database system is ready to accept connections
I20260811 14:32:10.762732 24299 mini_postgres.cc:96] Postgres bound to 50215
2026-08-11 14:32:10.770 UTC [5753] FATAL: database "slave" does not exist
127.23.186.212:50215 - accepting connections
I20260811 14:32:10.770924 24299 mini_ranger.cc:161] Starting Ranger...
I20260811 14:32:10.787438 24299 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 14:32:11.291817 24299 mini_ranger.cc:87] Created ranger Postgres database
I20260811 14:32:11.291911 24299 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-admin
2026-08-11 14:32:11,605 [I] DB FLAVOR :POSTGRES
2026-08-11 14:32:11,605 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:32:11,606 [I] Checking connection..
2026-08-11 14:32:11,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 14:32:11,882 [I] Checking connection passed.
2026-08-11 14:32:11,883 [I] --------- Verifying version history table ---------
2026-08-11 14:32:11,883 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:32:12,163 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:32:12,164 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:32:12,445 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 14:32:12,445 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 14:32:12,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 14:32:12.720 UTC [5830] WARNING: there is no transaction in progress
2026-08-11 14:32:12,730 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 14:32:12,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 14:32:13,037 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 14:32:13,037 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:32:13,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 14:32:13,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 14:32:13,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cbsf.c.gcp-upstream.internal','N') ;"
2026-08-11 14:32:13,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:32:14,229 [I] Table x_portal_user does not exist in database ranger
2026-08-11 14:32:14,229 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:32:14,554 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 14:32:14,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:32:14,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:32:15,146 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 14:32:15,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 14:32:16.504 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.525 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.548 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.569 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.591 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.613 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.685 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.694 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.715 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.729 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.745 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.764 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.780 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.790 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.802 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:16.813 UTC [5958] WARNING: there is no transaction in progress
2026-08-11 14:32:17,850 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 14:32:17,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 14:32:18,161 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 14:32:18,161 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 14:32:18,469 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 14:32:18,470 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 14:32:18,775 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 14:32:18,775 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 14:32:19,092 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 14:32:19,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cbsf.c.gcp-upstream.internal';"
2026-08-11 14:32:19,722 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 14:32:19,723 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 14:32:20,032 [I] DB_PATCHES have already been applied
I20260811 14:32:20.040113 24299 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLI13IK/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 14:32:20.040284 24299 mini_ranger.cc:193] Using host: 127.23.186.212
I20260811 14:32:20.046033 24299 mini_ranger.cc:227] Ranger admin URL: http://127.23.186.212:57923
Aug 11, 2026 2:32:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:32:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:32:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 2:32:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 2:32:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 2:32:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 2:32:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 14:32:21.435014 24299 mini_ranger.cc:160] Time spent starting Ranger: real 10.664s user 0.001s sys 0.014s
Aug 11, 2026 2:32:50 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 2:32:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 2:32:50 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 2:32:50 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 2:32:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 14:32:53.033843 24299 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.23.186.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.23.186.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/rangerkms_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.23.186.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/rangerkms_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.23.186.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.23.186.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.23.186.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/HTTP_127.23.186.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 14:32:53 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458773, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 14:32:53.188436 24299 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 14:32:53.207504 24299 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 14:32:54.323792 24299 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 14:32:54.323891 24299 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms
I20260811 14:32:54.323905 24299 mini_ranger_kms.cc:227] Using postgres at 127.23.186.212:50215
2026-08-11 14:32:54,535 [I] DB FLAVOR :POSTGRES
2026-08-11 14:32:54,536 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 14:32:54,536 [I] Checking connection
2026-08-11 14:32:54,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 14:32:54,870 [I] connection success
2026-08-11 14:32:54,870 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 14:32:54,870 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 14:32:54,870 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 14:32:55,198 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 14:32:55,198 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 14:32:55,198 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 14:32:55,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.23.186.212:50215/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 14:32:55,577 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 14:32:56.317236 24299 mini_ranger_kms.cc:310] Created kms service
I20260811 14:32:56.587255 24299 mini_ranger_kms.cc:326] Created kudu user
I20260811 14:32:56.650594 24299 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 14:32:56 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458773, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:32:57.177791 24299 mini_ranger_kms.cc:384] Added ranger policy
I20260811 14:32:57.178035 24299 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLI13IK/build/release/bin/postgresql.jar:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLI13IK/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 14:32:57.178056 24299 mini_ranger_kms.cc:240] Using host: 127.23.186.212
I20260811 14:32:57.184278 24299 mini_ranger_kms.cc:276] Ranger KMS PID: 6179
I20260811 14:32:57.184446 24299 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.23.186.212:54583
Aug 11, 2026 2:32:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 2:32:58 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 2:32:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 2:32:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 2:32:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 2:32:59 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 14:32:59.932411 24299 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.744s user 0.014s sys 0.180s
I20260811 14:32:59.932549 24299 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 14:32:59.932607 24299 mini_ranger_kms.cc:401] 127.23.186.212:54583/kms/v1/keys
Aug 11 14:33:00 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786458780, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.186.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 14:33:00 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458780, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.186.212@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
Aug 11 14:33:01 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458780, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.23.186.212@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
Aug 11 14:33:04 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:33:04.077056 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44153
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:44153
--ranger_config_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:04.177551 6243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:04.177853 6243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:04.177896 6243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:04.179205 6243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:04.179291 6243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:04.179316 6243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:04.179337 6243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:04.179369 6243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:04.181252 6243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
--ranger_kms_url=127.23.186.212:54583/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:44153
--ranger_config_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:04.181604 6243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:04.181878 6243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:04.188640 6252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:04.188640 6250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:04.188719 6249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:04.188684 6243 server_base.cc:1034] running on GCE node
I20260811 14:33:04.189327 6243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:04.189684 6243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:04.190956 6243 hybrid_clock.cc:648] HybridClock initialized: now 1786458784190906 us; error 65 us; skew 500 ppm
Aug 11 14:33:04 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458784, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:04.194759 6243 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:04.195369 6243 webserver.cc:460] Webserver started at http://127.23.186.254:38579/ using document root <none> and password file <none>
I20260811 14:33:04.195654 6243 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:04.195724 6243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:04.195897 6243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:33:04 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458784, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:04.259625 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "f6939ab4097849079648b31af80da2ba"
format_stamp: "Formatted at 2026-08-11 14:33:04 on dist-test-slave-cbsf"
server_key: "9126ffe4899f2f8a5fb0b071e712a406"
server_key_iv: "7c587d601e19663b996f33d679ac05cc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:04.260111 6243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "f6939ab4097849079648b31af80da2ba"
format_stamp: "Formatted at 2026-08-11 14:33:04 on dist-test-slave-cbsf"
server_key: "9126ffe4899f2f8a5fb0b071e712a406"
server_key_iv: "7c587d601e19663b996f33d679ac05cc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:04.261842 6243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260811 14:33:04.281065 6261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:04.281417 6243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:04.281566 6243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "f6939ab4097849079648b31af80da2ba"
format_stamp: "Formatted at 2026-08-11 14:33:04 on dist-test-slave-cbsf"
server_key: "9126ffe4899f2f8a5fb0b071e712a406"
server_key_iv: "7c587d601e19663b996f33d679ac05cc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:04.281738 6243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:04.300649 6243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:04.303730 6243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:04.304075 6243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:04.309537 6243 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:44153
I20260811 14:33:04.309566 6325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:44153 every 8 connection(s)
I20260811 14:33:04.310022 6243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:04.311223 6326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:04.314814 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Bootstrap starting.
I20260811 14:33:04.316124 6326 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:04.316650 6326 log.cc:824] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:04.317836 6326 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: No bootstrap required, opened a new log
I20260811 14:33:04.318176 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 6243
I20260811 14:33:04.318284 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:33:04.318464 24299 external_mini_cluster.cc:1428] Setting key bb0cd5cea3b505a0759a9a5bcd388e2c
I20260811 14:33:04.320858 6326 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:04.321079 6326 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:04.321137 6326 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6939ab4097849079648b31af80da2ba, State: Initialized, Role: FOLLOWER
I20260811 14:33:04.321326 6326 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:04.321434 6326 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:04.321508 6326 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:04.321571 6326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:04.322371 6326 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:04.322531 6326 leader_election.cc:302] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6939ab4097849079648b31af80da2ba; no voters:
I20260811 14:33:04.322757 6326 leader_election.cc:288] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:33:04.322795 6331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:04.322975 6331 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 LEADER]: Becoming Leader. State: Replica: f6939ab4097849079648b31af80da2ba, State: Running, Role: LEADER
I20260811 14:33:04.323081 6326 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:04.323136 6331 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:04.323616 6332 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6939ab4097849079648b31af80da2ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } } }
I20260811 14:33:04.323632 6333 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6939ab4097849079648b31af80da2ba. Latest consensus state: current_term: 1 leader_uuid: "f6939ab4097849079648b31af80da2ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } } }
I20260811 14:33:04.323736 6332 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:04.323753 6333 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:04.324805 6326 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 14:33:04.324934 6343 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:04.328595 6343 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 14:33:04 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458784, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:04.333823 6343 catalog_manager.cc:1347] Generated new cluster ID: 1a9e3d780e2d49fcab8163a79fc30205
I20260811 14:33:04.334234 6343 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:04.338541 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:04.319209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59098 (local address 127.23.186.254:44153)
0811 14:33:04.319434 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:04.319443 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:04.319478 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:33:04.320893 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:04.320912 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:04.321045 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:04.321405 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:04.322534 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:04.324219 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:04.327064 (+ 2845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:04.329144 (+ 2080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:04.329529 (+ 385us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:04.332980 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:04.332985 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:04.332986 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:04.333003 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:04.336034 (+ 3031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:04.336575 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:04.336579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:04.336580 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:04.336756 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:04.336991 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:04.336991 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:04.336992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:04.337414 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:04.337557 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:04.338109 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 14:33:04.338282 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":106,"threads_started":1}
I20260811 14:33:04.362427 6343 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:33:04.363061 6343 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:04.368033 6343 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Generated new TSK 0
I20260811 14:33:04.368276 6343 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:33:05.164204 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:44153
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:05.281292 6382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:05.281667 6382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:05.281716 6382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:05.283558 6382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:05.283669 6382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:05.283731 6382 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:05.286218 6382 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.23.186.212:54583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:44153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:05.286651 6382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:05.287079 6382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:05.293982 6392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:05.293975 6389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:05.294574 6390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:05.295913 6382 server_base.cc:1034] running on GCE node
I20260811 14:33:05.296308 6382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:05.296774 6382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:05.298056 6382 hybrid_clock.cc:648] HybridClock initialized: now 1786458785298016 us; error 51 us; skew 500 ppm
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:05.302448 6382 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:05.303159 6382 webserver.cc:460] Webserver started at http://127.23.186.193:45413/ using document root <none> and password file <none>
I20260811 14:33:05.303503 6382 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:05.303587 6382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:05.303783 6382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:05.326200 6382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "84c9e2ebc1f14cb2860608f00cde4fec"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "5a55a3337a01604106d002f7e8c8147c"
server_key_iv: "503dc4810c4db4433839bab0d96b7ee9"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:05.327210 6382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "84c9e2ebc1f14cb2860608f00cde4fec"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "5a55a3337a01604106d002f7e8c8147c"
server_key_iv: "503dc4810c4db4433839bab0d96b7ee9"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:05.329706 6382 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 14:33:05.351956 6399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:05.352409 6382 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 14:33:05.352597 6382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "84c9e2ebc1f14cb2860608f00cde4fec"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "5a55a3337a01604106d002f7e8c8147c"
server_key_iv: "503dc4810c4db4433839bab0d96b7ee9"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:05.352761 6382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:05.368021 6382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:05.371184 6382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:05.371434 6382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:05.371803 6382 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:05.372306 6382 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:05.372376 6382 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:05.372413 6382 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:05.372467 6382 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:05.381733 6382 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:34131
I20260811 14:33:05.382377 6382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:05.382409 6532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:34131 every 8 connection(s)
I20260811 14:33:05.383848 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 6382
I20260811 14:33:05.383971 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:33:05.384174 24299 external_mini_cluster.cc:1428] Setting key 707f8919502b4a6b2cfa28ddc2e23e56
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:05.398785 6535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:05.383886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44153 (local address 127.23.186.193:40667)
0811 14:33:05.384302 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:05.384362 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:05.385147 (+ 785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:05.385593 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:05.385598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:05.385884 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:05.386230 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:05.386238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.387331 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.387332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:05.388769 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:05.388774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.389400 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.389401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:05.389575 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:05.390330 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:05.390343 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:05.392897 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:05.396241 (+ 3344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:05.396264 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:05.396269 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:05.396864 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:05.397237 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:05.397238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:05.397239 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:05.397428 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:05.397816 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:05.397824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:05.398076 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 14:33:05.398204 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 14:33:05.398439 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":101,"threads_started":1}
I20260811 14:33:05.399242 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:05.383952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:40667 (local address 127.23.186.254:44153)
0811 14:33:05.385352 (+ 1400us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:05.385354 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:05.385363 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:05.385389 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:05.385390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:05.385403 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:05.386343 (+ 940us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:05.386394 (+ 51us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.387196 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.388898 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.389219 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.389296 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:05.393158 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:05.393161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:05.393162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:05.393178 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:05.396039 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:05.396987 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:05.396990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:05.396990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:05.397087 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:05.397514 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:05.397515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:05.397515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:05.397716 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:05.397786 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:05.399009 (+ 1223us) server_negotiation.cc:299] Negotiation successful
0811 14:33:05.399136 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1372,"thread_start_us":85,"threads_started":1}
I20260811 14:33:05.400031 6533 heartbeater.cc:347] Connected to a master server at 127.23.186.254:44153
I20260811 14:33:05.400247 6533 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:05.400544 6533 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:05.401542 6282 ts_manager.cc:194] Registered new tserver with Master: 84c9e2ebc1f14cb2860608f00cde4fec (127.23.186.193:34131)
I20260811 14:33:05.402498 6282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:40667
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:33:05.440740 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:44153
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:05.585318 6544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:05.585868 6544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:05.585983 6544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:05.588979 6544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:05.589401 6544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:05.589524 6544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:05.592890 6544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.23.186.212:54583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:44153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:05.593652 6544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:05.594086 6544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:05.615525 6555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:05.615931 6554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:05.616199 6558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:05.627004 6544 server_base.cc:1034] running on GCE node
I20260811 14:33:05.628026 6544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:05.628523 6544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:05.635980 6544 hybrid_clock.cc:648] HybridClock initialized: now 1786458785635916 us; error 102 us; skew 500 ppm
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:05.642059 6544 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:05.642918 6544 webserver.cc:460] Webserver started at http://127.23.186.194:42661/ using document root <none> and password file <none>
I20260811 14:33:05.643321 6544 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:05.643452 6544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:05.643651 6544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:05.670642 6544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "2a6110c574874e22b02917ef6ad6a128"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "693f2d690c8b556740910fdea6536e08"
server_key_iv: "f924d518f7538350f29f2cfce8b1b4dc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:05.674170 6544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "2a6110c574874e22b02917ef6ad6a128"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "693f2d690c8b556740910fdea6536e08"
server_key_iv: "f924d518f7538350f29f2cfce8b1b4dc"
server_key_version: "kuduclusterkey@0"
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:05.688823 6544 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.000s
I20260811 14:33:05.734278 6565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:05.734720 6544 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.000s sys 0.000s
I20260811 14:33:05.734882 6544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "2a6110c574874e22b02917ef6ad6a128"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "693f2d690c8b556740910fdea6536e08"
server_key_iv: "f924d518f7538350f29f2cfce8b1b4dc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:05.735049 6544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:05.769094 6544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:05.772392 6544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:05.772575 6544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:05.772912 6544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:05.773481 6544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:05.773571 6544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:05.773648 6544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:05.773711 6544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:05.783208 6544 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:34463
I20260811 14:33:05.783843 6544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:33:05.784736 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 6544
I20260811 14:33:05.784879 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:33:05.785115 24299 external_mini_cluster.cc:1428] Setting key 4315074326a17f4d6abb25f48c794422
I20260811 14:33:05.785235 6701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:34463 every 8 connection(s)
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:05.819196 6705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:05.799217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44153 (local address 127.23.186.194:43183)
0811 14:33:05.804685 (+ 5468us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:05.805402 (+ 717us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:05.806247 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:05.806749 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:05.806754 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:05.807184 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:05.807539 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:05.807546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.808596 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.808598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:05.810148 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:05.810152 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.811958 (+ 1806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.811959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:05.812190 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:05.812964 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:05.812977 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:05.815044 (+ 2067us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:05.817308 (+ 2264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:05.817321 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:05.817325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:05.817687 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:05.817923 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:05.817924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:05.817925 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:05.818093 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:05.818414 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:05.818421 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:05.818616 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 14:33:05.818723 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 14:33:05.818911 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5361,"thread_start_us":116,"threads_started":1}
I20260811 14:33:05.819584 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:05.799617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:43183 (local address 127.23.186.254:44153)
0811 14:33:05.799677 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:05.799679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:05.805561 (+ 5882us) server_negotiation.cc:407] Connection header received
0811 14:33:05.806428 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:05.806429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:05.806475 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:05.806510 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:05.807674 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.808439 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.810271 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:05.810568 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:05.810634 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:05.815203 (+ 4569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:05.815206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:05.815207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:05.815217 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:05.817180 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:05.817785 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:05.817786 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:05.817787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:05.817855 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:05.818166 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:05.818167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:05.818167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:05.818326 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:05.818388 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:05.819371 (+ 983us) server_negotiation.cc:299] Negotiation successful
0811 14:33:05.819473 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:33:05.820060 6702 heartbeater.cc:347] Connected to a master server at 127.23.186.254:44153
I20260811 14:33:05.820171 6702 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:05.820396 6702 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:05.820950 6282 ts_manager.cc:194] Registered new tserver with Master: 2a6110c574874e22b02917ef6ad6a128 (127.23.186.194:34463)
I20260811 14:33:05.821460 6282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:43183
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:33:05.832949 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:44153
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:05.966573 6709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:05.967048 6709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:05.967166 6709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:05.969635 6709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:05.969973 6709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:05.970265 6709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:05.973220 6709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.23.186.212:54583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:44153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:05.973850 6709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:05.974267 6709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:33:05.983287 6709 server_base.cc:1034] running on GCE node
W20260811 14:33:05.984138 6719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:05.985325 6717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:05.985364 6716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:05.985977 6709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:05.986438 6709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:05.987699 6709 hybrid_clock.cc:648] HybridClock initialized: now 1786458785987676 us; error 56 us; skew 500 ppm
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:05.992833 6709 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:05.993573 6709 webserver.cc:460] Webserver started at http://127.23.186.195:40347/ using document root <none> and password file <none>
I20260811 14:33:05.993949 6709 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:05.994086 6709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:05.994282 6709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 14:33:05 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:06.019248 6709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "48a7836b306d4d18be80ce4ef90f2357"
format_stamp: "Formatted at 2026-08-11 14:33:06 on dist-test-slave-cbsf"
server_key: "32cec9e613e6fdd48345e6ddd3842e2b"
server_key_iv: "02f47b98905cfd2c7df2183cdd79a35d"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:06.019958 6709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "48a7836b306d4d18be80ce4ef90f2357"
format_stamp: "Formatted at 2026-08-11 14:33:06 on dist-test-slave-cbsf"
server_key: "32cec9e613e6fdd48345e6ddd3842e2b"
server_key_iv: "02f47b98905cfd2c7df2183cdd79a35d"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:06.029245 6709 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260811 14:33:06.050722 6729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.051352 6709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260811 14:33:06.051497 6709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "48a7836b306d4d18be80ce4ef90f2357"
format_stamp: "Formatted at 2026-08-11 14:33:06 on dist-test-slave-cbsf"
server_key: "32cec9e613e6fdd48345e6ddd3842e2b"
server_key_iv: "02f47b98905cfd2c7df2183cdd79a35d"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:06.051617 6709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:06.069620 6709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:06.074184 6709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:06.074440 6709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:06.075081 6709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:06.075582 6709 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:06.077134 6709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:33:06.077221 6709 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:06.077248 6709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.088613 6709 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:39049
I20260811 14:33:06.089265 6709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:33:06.090534 6863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:39049 every 8 connection(s)
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458785, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:06.098044 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 6709
I20260811 14:33:06.098227 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:33:06.098469 24299 external_mini_cluster.cc:1428] Setting key 18e4e3cc39ccd7fea96fccf7f9ae0401
I20260811 14:33:06.102428 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:06.091246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:52195 (local address 127.23.186.254:44153)
0811 14:33:06.091298 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:06.091300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:06.091579 (+ 279us) server_negotiation.cc:407] Connection header received
0811 14:33:06.092187 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:06.092188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:06.092202 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:06.092250 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:06.093231 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.093772 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.095274 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.095484 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.095595 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:06.098165 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:06.098167 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:06.098168 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:06.098176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:06.100481 (+ 2305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:06.101160 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:06.101161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:06.101162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:06.101245 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:06.101552 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:06.101554 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:06.101554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:06.101718 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:06.101798 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:06.102179 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 14:33:06.102311 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:33:06.102682 6868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:06.091042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44153 (local address 127.23.186.195:52195)
0811 14:33:06.091409 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:06.091461 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:06.092062 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:06.092326 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:06.092331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:06.092604 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:06.092915 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:06.093092 (+ 177us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.093881 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.093883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:06.095041 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:06.095045 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.095612 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.095613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:06.095759 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:06.096372 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:06.096382 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:06.097976 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:06.100608 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:06.100626 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:06.100631 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:06.101064 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:06.101321 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:06.101322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:06.101323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:06.101475 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:06.101795 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:06.101805 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:06.102006 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 14:33:06.102140 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 14:33:06.102367 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":179,"threads_started":1}
I20260811 14:33:06.103259 6865 heartbeater.cc:347] Connected to a master server at 127.23.186.254:44153
I20260811 14:33:06.103399 6865 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.103652 6865 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:06.104204 6282 ts_manager.cc:194] Registered new tserver with Master: 48a7836b306d4d18be80ce4ef90f2357 (127.23.186.195:39049)
I20260811 14:33:06.104842 6282 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:52195
I20260811 14:33:06.112399 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:33:06.113494 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 6382
I20260811 14:33:06.126577 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 6544
I20260811 14:33:06.138657 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 6709
I20260811 14:33:06.152628 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 6243
I20260811 14:33:06.164031 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44153
--webserver_interface=127.23.186.254
--webserver_port=38579
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:44153
--ranger_config_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:06.266235 6869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:06.266530 6869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:06.266557 6869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:06.267820 6869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:06.267877 6869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:06.267895 6869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:06.267910 6869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:06.267925 6869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:06.269842 6869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:44153
--ranger_config_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=38579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:06.270243 6869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:06.270491 6869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:06.276103 6879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:06.276126 6877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:06.276152 6869 server_base.cc:1034] running on GCE node
W20260811 14:33:06.276113 6875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:06.276664 6869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:06.277035 6869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:06.278231 6869 hybrid_clock.cc:648] HybridClock initialized: now 1786458786278208 us; error 35 us; skew 500 ppm
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:06.281280 6869 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:06.282038 6869 webserver.cc:460] Webserver started at http://127.23.186.254:38579/ using document root <none> and password file <none>
I20260811 14:33:06.282330 6869 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:06.282413 6869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:06.364938 6869 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 14:33:06.366317 6889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.366729 6869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:06.366874 6869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "f6939ab4097849079648b31af80da2ba"
format_stamp: "Formatted at 2026-08-11 14:33:04 on dist-test-slave-cbsf"
server_key: "9126ffe4899f2f8a5fb0b071e712a406"
server_key_iv: "7c587d601e19663b996f33d679ac05cc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:06.367569 6869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:06.386756 6869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:06.389930 6869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:06.390120 6869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:06.396205 6869 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:44153
I20260811 14:33:06.396232 6947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:44153 every 8 connection(s)
I20260811 14:33:06.396755 6869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:06.398192 6948 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:33:06.398914 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Bootstrap starting.
I20260811 14:33:06.404493 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 6869
I20260811 14:33:06.405030 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:34131
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:44153
--webserver_port=45413
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:06.410012 6948 log.cc:824] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:06.412674 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:06.413148 6948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Bootstrap complete.
I20260811 14:33:06.415933 6948 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:06.416268 6948 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6939ab4097849079648b31af80da2ba, State: Initialized, Role: FOLLOWER
I20260811 14:33:06.416472 6948 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:06.416569 6948 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:06.416630 6948 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:06.416683 6948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:33:06.418018 6948 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:06.418154 6948 leader_election.cc:302] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6939ab4097849079648b31af80da2ba; no voters:
I20260811 14:33:06.418375 6948 leader_election.cc:288] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:33:06.418589 6957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:33:06.418682 6948 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:06.420222 6957 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [term 2 LEADER]: Becoming Leader. State: Replica: f6939ab4097849079648b31af80da2ba, State: Running, Role: LEADER
I20260811 14:33:06.420452 6957 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } }
I20260811 14:33:06.421094 6967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6939ab4097849079648b31af80da2ba. Latest consensus state: current_term: 2 leader_uuid: "f6939ab4097849079648b31af80da2ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } } }
I20260811 14:33:06.421207 6967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:06.421289 6956 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f6939ab4097849079648b31af80da2ba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6939ab4097849079648b31af80da2ba" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44153 } } }
I20260811 14:33:06.421476 6956 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:06.421789 6973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:06.422381 6948 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 14:33:06.422441 6973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:06.422603 6973 catalog_manager.cc:1259] Loaded cluster ID: 1a9e3d780e2d49fcab8163a79fc30205
I20260811 14:33:06.422649 6973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:06.428169 6973 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:06.429008 6973 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Loaded TSK: 0
I20260811 14:33:06.429378 6973 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:33:06.518050 6950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:06.518424 6950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:06.518474 6950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:06.520874 6950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:06.520982 6950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:06.521044 6950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:06.523473 6950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:34131
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=45413
--tserver_master_addrs=127.23.186.254:44153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:06.523905 6950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:06.524245 6950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:06.532143 7001 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:06.532413 6998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:06.532617 6950 server_base.cc:1034] running on GCE node
W20260811 14:33:06.532482 6999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:06.533130 6950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:06.533456 6950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:06.536805 6950 hybrid_clock.cc:648] HybridClock initialized: now 1786458786536280 us; error 529 us; skew 500 ppm
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:06.539852 6950 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:06.540496 6950 webserver.cc:460] Webserver started at http://127.23.186.193:45413/ using document root <none> and password file <none>
I20260811 14:33:06.540831 6950 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:06.540925 6950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:06.564181 6950 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:06.565418 7010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.565656 6950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:06.565768 6950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "84c9e2ebc1f14cb2860608f00cde4fec"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "5a55a3337a01604106d002f7e8c8147c"
server_key_iv: "503dc4810c4db4433839bab0d96b7ee9"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:06.566244 6950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:06.580534 6950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:06.583492 6950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:06.583691 6950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:06.584014 6950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:06.584450 6950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:06.584517 6950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.584558 6950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:06.584583 6950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.594055 6950 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:34131
I20260811 14:33:06.594696 6950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:06.601728 7140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:34131 every 8 connection(s)
I20260811 14:33:06.605048 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 6950
I20260811 14:33:06.605907 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:34463
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:44153
--webserver_port=42661
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:06.624521 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:06.596093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44153 (local address 127.23.186.193:44865)
0811 14:33:06.603500 (+ 7407us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:06.603559 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:06.604309 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:06.605167 (+ 858us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:06.605173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:06.605445 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:06.605768 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:06.605775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.607608 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.607610 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:06.609019 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:06.609024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.610120 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.610122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:06.610273 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:06.615144 (+ 4871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:06.615161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:06.617541 (+ 2380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:06.621655 (+ 4114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:06.621683 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:06.621688 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:06.622193 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:06.622595 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:06.622597 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:06.622597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:06.622784 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:06.623516 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:06.623528 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:06.623769 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 14:33:06.623910 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 14:33:06.624162 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7340,"thread_start_us":100,"threads_started":1}
I20260811 14:33:06.626451 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:06.596158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:44865 (local address 127.23.186.254:44153)
0811 14:33:06.603737 (+ 7579us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:06.603751 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:06.603809 (+ 58us) server_negotiation.cc:407] Connection header received
0811 14:33:06.604737 (+ 928us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:06.604747 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:06.604817 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:06.604968 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:06.605919 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.607445 (+ 1526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.609143 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.609836 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.609961 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:06.617927 (+ 7966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:06.617938 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:06.617939 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:06.617964 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:06.621432 (+ 3468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:06.622336 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:06.622342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:06.622343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:06.622506 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:06.622876 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:06.622877 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:06.622878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:06.623345 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:06.623472 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:06.625813 (+ 2341us) server_negotiation.cc:299] Negotiation successful
0811 14:33:06.626104 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7072,"thread_start_us":92,"threads_started":1}
I20260811 14:33:06.628196 7141 heartbeater.cc:347] Connected to a master server at 127.23.186.254:44153
I20260811 14:33:06.628417 7141 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.628844 7141 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
W20260811 14:33:06.630334 7141 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:44153 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 14:33:06.630455 7141 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.630571 7141 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:06.631059 7141 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.631167 7141 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
W20260811 14:33:06.631474 7141 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:33:06.772007 7145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:06.772408 7145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:06.772459 7145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:06.774614 7145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:06.774708 7145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:06.774756 7145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:06.777587 7145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:34463
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=42661
--tserver_master_addrs=127.23.186.254:44153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:06.778311 7145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:06.778797 7145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:06.786513 7151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:06.786513 7152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:06.787111 7154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:06.787817 7145 server_base.cc:1034] running on GCE node
I20260811 14:33:06.788156 7145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:06.788607 7145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:06.789861 7145 hybrid_clock.cc:648] HybridClock initialized: now 1786458786789830 us; error 55 us; skew 500 ppm
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:06.794507 7145 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:06.795426 7145 webserver.cc:460] Webserver started at http://127.23.186.194:42661/ using document root <none> and password file <none>
I20260811 14:33:06.795849 7145 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:06.795987 7145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:06.822376 7145 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:06.823455 7162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.823690 7145 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:06.823794 7145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "2a6110c574874e22b02917ef6ad6a128"
format_stamp: "Formatted at 2026-08-11 14:33:05 on dist-test-slave-cbsf"
server_key: "693f2d690c8b556740910fdea6536e08"
server_key_iv: "f924d518f7538350f29f2cfce8b1b4dc"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:06.824321 7145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:06.845403 7145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:06.848268 7145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:06.848467 7145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:06.848809 7145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:06.849277 7145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:06.849334 7145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.849375 7145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:06.849399 7145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:06.857501 7145 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:34463
I20260811 14:33:06.858146 7145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:33:06.861204 7296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:34463 every 8 connection(s)
I20260811 14:33:06.865429 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 7145
I20260811 14:33:06.865891 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.23.186.212:54583/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:39049
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:44153
--webserver_port=40347
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41535
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 14:33:06 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458786, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:06.874888 7300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:06.862002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44153 (local address 127.23.186.194:48881)
0811 14:33:06.862256 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:06.862307 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:06.862925 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:06.863241 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:06.863245 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:06.863483 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:06.863722 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:06.863726 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.864540 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.864541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:06.865601 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:06.865604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.866131 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.866132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:06.866275 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:06.866916 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:06.866928 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:06.869576 (+ 2648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:06.872319 (+ 2743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:06.872340 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:06.872345 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:06.872876 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:06.873244 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:06.873245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:06.873245 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:06.873434 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:06.873805 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:06.873815 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:06.874074 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 14:33:06.874216 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 14:33:06.874488 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":168,"threads_started":1}
I20260811 14:33:06.875653 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:06.861981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:48881 (local address 127.23.186.254:44153)
0811 14:33:06.862040 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:06.862043 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:06.862431 (+ 388us) server_negotiation.cc:407] Connection header received
0811 14:33:06.863053 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:06.863054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:06.863074 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:06.863130 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:06.863842 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.864424 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.865708 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:06.866012 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:06.866098 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:06.869808 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:06.869811 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:06.869812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:06.869828 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:06.872170 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:06.872999 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:06.873001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:06.873002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:06.873115 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:06.873510 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:06.873511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:06.873512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:06.873709 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:06.873775 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:06.875348 (+ 1573us) server_negotiation.cc:299] Negotiation successful
0811 14:33:06.875512 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:33:06.876191 7297 heartbeater.cc:347] Connected to a master server at 127.23.186.254:44153
I20260811 14:33:06.876353 7297 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.876724 7297 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
W20260811 14:33:06.877738 7297 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:44153 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 14:33:06.877846 7297 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.877974 7297 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:06.878419 7297 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:06.878535 7297 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
W20260811 14:33:06.878891 7297 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:33:07.008906 7301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:07.009267 7301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:07.009315 7301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:07.011643 7301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:07.011725 7301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:07.011777 7301 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:07.014827 7301 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41535
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.23.186.212:54583/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:39049
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=40347
--tserver_master_addrs=127.23.186.254:44153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:07.015220 7301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:07.015570 7301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:07.025326 7306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:07.025326 7307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:07.027109 7301 server_base.cc:1034] running on GCE node
W20260811 14:33:07.027957 7309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:07.028380 7301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:07.028838 7301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:07.030037 7301 hybrid_clock.cc:648] HybridClock initialized: now 1786458787030005 us; error 55 us; skew 500 ppm
Aug 11 14:33:07 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458787, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:07.034343 7301 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:07.035045 7301 webserver.cc:460] Webserver started at http://127.23.186.195:40347/ using document root <none> and password file <none>
I20260811 14:33:07.035352 7301 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:07.035445 7301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 14:33:07 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458787, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:07.060441 7301 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260811 14:33:07.061777 7318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:07.062017 7301 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:07.062179 7301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "48a7836b306d4d18be80ce4ef90f2357"
format_stamp: "Formatted at 2026-08-11 14:33:06 on dist-test-slave-cbsf"
server_key: "32cec9e613e6fdd48345e6ddd3842e2b"
server_key_iv: "02f47b98905cfd2c7df2183cdd79a35d"
server_key_version: "kuduclusterkey@0"
I20260811 14:33:07.062740 7301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:07.081458 7301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:07.084357 7301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:07.084542 7301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:07.084880 7301 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:07.085340 7301 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:07.085386 7301 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:07.085418 7301 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:07.085436 7301 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:07.095065 7301 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:39049
I20260811 14:33:07.095602 7301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:33:07.096443 7448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:39049 every 8 connection(s)
I20260811 14:33:07.100394 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 7301
Aug 11 14:33:07 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458787, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:07.110680 7144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:07.096745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:34131 (local address 127.23.186.254:44153)
0811 14:33:07.096807 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:07.096810 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:07.097315 (+ 505us) server_negotiation.cc:407] Connection header received
0811 14:33:07.097923 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:07.097925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:07.097940 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:07.098022 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:07.098630 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:07.099402 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:07.101177 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:07.101500 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:07.101546 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:07.105584 (+ 4038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:07.105585 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:07.105585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:07.105593 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:07.108003 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:07.109055 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:07.109057 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:07.109057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:07.109124 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:07.109473 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:07.109474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:07.109474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:07.109637 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:07.109703 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:07.110498 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 14:33:07.110602 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:33:07.111536 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:07.096750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44153 (local address 127.23.186.195:34131)
0811 14:33:07.097182 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:07.097218 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:07.097781 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:07.098109 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:07.098113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:07.098311 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:07.098524 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:07.098528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:07.099530 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:07.099532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:07.101067 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:07.101072 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:07.101672 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:07.101674 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:07.101848 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:07.102582 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:07.102595 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:07.104546 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:07.108116 (+ 3570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:07.108134 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:07.108139 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:07.108938 (+ 799us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:07.109185 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:07.109185 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:07.109186 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:07.109370 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:07.109752 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:07.109760 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:07.109979 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 14:33:07.110982 (+ 1003us) client_negotiation.cc:240] Negotiation successful
0811 14:33:07.111225 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":119,"threads_started":1}
I20260811 14:33:07.112090 7449 heartbeater.cc:347] Connected to a master server at 127.23.186.254:44153
I20260811 14:33:07.112228 7449 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:07.112454 7449 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
W20260811 14:33:07.113233 7449 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:44153 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 14:33:07.113322 7449 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:07.113400 7449 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:07.113726 7449 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:07.113803 7449 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:07.113925 7454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:07.101177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59102 (local address 127.23.186.254:44153)
0811 14:33:07.101480 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:07.101481 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:07.101488 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:33:07.101509 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:07.101510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:07.101523 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:07.101603 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:07.102520 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:07.103321 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:07.104759 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:07.105175 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:07.105438 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:07.108123 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:07.108125 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:07.108125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:07.108134 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:07.110027 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:07.110408 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:07.110410 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:07.110410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:07.110473 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:07.111173 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:07.111174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:07.111175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:07.111332 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:07.111519 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:07.112831 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0811 14:33:07.113066 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":69,"threads_started":1}
W20260811 14:33:07.114817 7449 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:33:07.631810 7141 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:07.632094 7141 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:07.633467 6908 ts_manager.cc:194] Registered new tserver with Master: 84c9e2ebc1f14cb2860608f00cde4fec (127.23.186.193:34131)
I20260811 14:33:07.634407 6908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:44865
Aug 11 14:33:07 dist-test-slave-cbsf krb5kdc[5698](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786458787, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:07.879173 7297 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:07.879438 7297 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:07.879931 6908 ts_manager.cc:194] Registered new tserver with Master: 2a6110c574874e22b02917ef6ad6a128 (127.23.186.194:34463)
I20260811 14:33:07.880573 6908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:48881
I20260811 14:33:08.115144 7449 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:08.115361 7449 heartbeater.cc:511] Master 127.23.186.254:44153 requested a full tablet report, sending...
I20260811 14:33:08.115810 6908 ts_manager.cc:194] Registered new tserver with Master: 48a7836b306d4d18be80ce4ef90f2357 (127.23.186.195:39049)
I20260811 14:33:08.116346 6908 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:34131
I20260811 14:33:08.130734 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:33:08.158438 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.133177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59110 (local address 127.23.186.254:44153)
0811 14:33:08.143703 (+ 10526us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.143706 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.143715 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:08.143749 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.143750 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.143764 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.143846 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:08.144794 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.145782 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.147070 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.147691 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.147784 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.154497 (+ 6713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:08.154499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:08.154500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:08.154512 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:08.156823 (+ 2311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:08.157391 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:08.157393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:08.157393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:08.157492 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:08.157737 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:08.157738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:08.157738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:08.157925 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:08.158047 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.158206 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.158319 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10494,"thread_start_us":105,"threads_started":1}
I20260811 14:33:08.162153 6908 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:33:08.165817 6908 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:33:08.180368 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.174763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55716 (local address 127.23.186.194:34463)
0811 14:33:08.175283 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.175290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.175302 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:33:08.175345 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.175347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.175366 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.175479 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:08.175701 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.177041 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.178706 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.179532 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.179627 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.179696 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.180219 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.180280 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":484,"thread_start_us":71,"threads_started":1}
I20260811 14:33:08.180555 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.174211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:34463 (local address 127.0.0.1:55716)
0811 14:33:08.174968 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:08.175016 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:08.175093 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:08.175507 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:08.175511 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:08.175514 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:08.175599 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.175601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.177173 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.177174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.178589 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.178593 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.179737 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.179738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.179836 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.179863 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 14:33:08.179941 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 14:33:08.180003 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":716,"thread_start_us":126,"threads_started":1}
I20260811 14:33:08.180822 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.174221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:39049 (local address 127.0.0.1:59630)
0811 14:33:08.175702 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:08.175713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:08.175772 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:08.176040 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:08.176041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:08.176044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:08.176093 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.176094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.177831 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.177831 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.179026 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.179030 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.180550 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.180552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.180617 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.180622 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:33:08.180676 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 14:33:08.180743 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1457,"thread_start_us":130,"threads_started":1}
I20260811 14:33:08.181161 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.174572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.23.186.195:39049)
0811 14:33:08.175634 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.175639 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.175827 (+ 188us) server_negotiation.cc:407] Connection header received
0811 14:33:08.175867 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.175869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.175886 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.175959 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:08.176170 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.177726 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.179142 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.180348 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.180449 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.180509 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.181028 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.181077 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1028,"thread_start_us":77,"threads_started":1}
I20260811 14:33:08.181505 7219 tablet_service.cc:1467] Processing CreateTablet for tablet 8c280f1decfc41b9881580d3209e38fd (DEFAULT_TABLE table=test-table [id=50f74a1efe67473083381b6128a58974]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:08.181914 7219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c280f1decfc41b9881580d3209e38fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:08.181842 7377 tablet_service.cc:1467] Processing CreateTablet for tablet 8c280f1decfc41b9881580d3209e38fd (DEFAULT_TABLE table=test-table [id=50f74a1efe67473083381b6128a58974]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:08.182251 7377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c280f1decfc41b9881580d3209e38fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:08.186779 7487 tablet_bootstrap.cc:492] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Bootstrap starting.
I20260811 14:33:08.187681 7487 tablet_bootstrap.cc:654] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:08.188098 7487 log.cc:824] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:08.188632 7488 tablet_bootstrap.cc:492] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Bootstrap starting.
I20260811 14:33:08.189483 7488 tablet_bootstrap.cc:654] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:08.189926 7488 log.cc:824] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:08.190982 7487 tablet_bootstrap.cc:492] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: No bootstrap required, opened a new log
I20260811 14:33:08.191107 7487 ts_tablet_manager.cc:1397] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 14:33:08.192133 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.174525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:34131 (local address 127.0.0.1:41088)
0811 14:33:08.180959 (+ 6434us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:08.180973 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:08.181019 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:08.187322 (+ 6303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:08.187324 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:08.187327 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:08.187390 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.187392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.188934 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.188936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.190629 (+ 1693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.190634 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.191846 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.191847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.191918 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.191923 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:33:08.191983 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:33:08.192049 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6414,"thread_start_us":93,"threads_started":1}
I20260811 14:33:08.192521 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.174765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.23.186.193:34131)
0811 14:33:08.187068 (+ 12303us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.187085 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.187102 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:33:08.187152 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.187154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.187178 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.187299 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:08.187509 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.188798 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.190749 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.191641 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.191735 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.191800 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.192362 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.192421 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12253,"thread_start_us":76,"threads_started":1}
I20260811 14:33:08.193480 7064 tablet_service.cc:1467] Processing CreateTablet for tablet 8c280f1decfc41b9881580d3209e38fd (DEFAULT_TABLE table=test-table [id=50f74a1efe67473083381b6128a58974]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:08.193917 7064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c280f1decfc41b9881580d3209e38fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:08.194029 7487 raft_consensus.cc:348] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.194237 7487 raft_consensus.cc:374] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:08.194279 7487 raft_consensus.cc:729] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a7836b306d4d18be80ce4ef90f2357, State: Initialized, Role: FOLLOWER
I20260811 14:33:08.194439 7487 consensus_queue.cc:260] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.194774 7487 ts_tablet_manager.cc:1428] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260811 14:33:08.194905 7449 heartbeater.cc:503] Master 127.23.186.254:44153 was elected leader, sending a full tablet report...
I20260811 14:33:08.197175 7488 tablet_bootstrap.cc:492] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: No bootstrap required, opened a new log
I20260811 14:33:08.197330 7488 ts_tablet_manager.cc:1397] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 14:33:08.198145 7494 tablet_bootstrap.cc:492] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: Bootstrap starting.
I20260811 14:33:08.199235 7494 tablet_bootstrap.cc:654] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:08.199712 7494 log.cc:824] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:08.200472 7488 raft_consensus.cc:348] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.200999 7488 raft_consensus.cc:374] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:08.201066 7488 raft_consensus.cc:729] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a6110c574874e22b02917ef6ad6a128, State: Initialized, Role: FOLLOWER
I20260811 14:33:08.201295 7488 consensus_queue.cc:260] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.201748 7488 ts_tablet_manager.cc:1428] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 14:33:08.201941 7297 heartbeater.cc:503] Master 127.23.186.254:44153 was elected leader, sending a full tablet report...
I20260811 14:33:08.202589 7494 tablet_bootstrap.cc:492] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: No bootstrap required, opened a new log
I20260811 14:33:08.202708 7494 ts_tablet_manager.cc:1397] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 14:33:08.205812 7494 raft_consensus.cc:348] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.206143 7494 raft_consensus.cc:374] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:08.206192 7494 raft_consensus.cc:729] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84c9e2ebc1f14cb2860608f00cde4fec, State: Initialized, Role: FOLLOWER
I20260811 14:33:08.206445 7494 consensus_queue.cc:260] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.207243 7494 ts_tablet_manager.cc:1428] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 14:33:08.207590 7141 heartbeater.cc:503] Master 127.23.186.254:44153 was elected leader, sending a full tablet report...
Aug 11 14:33:08 dist-test-slave-cbsf krb5kdc[5698](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458787, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for HTTP/127.23.186.212@KRBTEST.COM
I20260811 14:33:08.330008 7492 raft_consensus.cc:482] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:08.330188 7492 raft_consensus.cc:504] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.330654 7492 leader_election.cc:288] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2a6110c574874e22b02917ef6ad6a128 (127.23.186.194:34463), 84c9e2ebc1f14cb2860608f00cde4fec (127.23.186.193:34131)
I20260811 14:33:08.340096 7486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.331223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:58007 (local address 127.23.186.194:34463)
0811 14:33:08.331309 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.331312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.336196 (+ 4884us) server_negotiation.cc:407] Connection header received
0811 14:33:08.336328 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.336330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.336341 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.336371 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:08.336629 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.337450 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.339035 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.339653 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.339709 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.339720 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.339968 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.340012 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:33:08.340382 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.330872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:34463 (local address 127.23.186.195:58007)
0811 14:33:08.336093 (+ 5221us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:08.336111 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:08.336239 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:08.336463 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:08.336464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:08.336466 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:08.336553 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.336555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.337578 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.337579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.338906 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.338910 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.339809 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.339810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.339884 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.339888 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:33:08.340232 (+ 344us) client_negotiation.cc:240] Negotiation successful
0811 14:33:08.340281 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5186,"thread_start_us":166,"threads_started":1}
I20260811 14:33:08.340827 7240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c280f1decfc41b9881580d3209e38fd" candidate_uuid: "48a7836b306d4d18be80ce4ef90f2357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6110c574874e22b02917ef6ad6a128" is_pre_election: true
I20260811 14:33:08.341022 7240 raft_consensus.cc:2454] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48a7836b306d4d18be80ce4ef90f2357 in term 0.
I20260811 14:33:08.341286 7319 leader_election.cc:302] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6110c574874e22b02917ef6ad6a128, 48a7836b306d4d18be80ce4ef90f2357; no voters:
I20260811 14:33:08.341461 7492 raft_consensus.cc:2781] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:33:08.341523 7492 raft_consensus.cc:482] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:33:08.341547 7492 raft_consensus.cc:3037] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:08.342168 7485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.331178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:43567 (local address 127.23.186.193:34131)
0811 14:33:08.331309 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.331312 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.337828 (+ 6516us) server_negotiation.cc:407] Connection header received
0811 14:33:08.337952 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.337953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.337965 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.337997 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:08.338233 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.338978 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.340580 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.341403 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.341520 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.341541 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.341954 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.342063 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 14:33:08.342563 7492 raft_consensus.cc:504] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.342779 7492 leader_election.cc:288] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [CANDIDATE]: Term 1 election: Requested vote from peers 2a6110c574874e22b02917ef6ad6a128 (127.23.186.194:34463), 84c9e2ebc1f14cb2860608f00cde4fec (127.23.186.193:34131)
I20260811 14:33:08.342577 7498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.330921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:34131 (local address 127.23.186.195:43567)
0811 14:33:08.337731 (+ 6810us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:08.337748 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:08.337867 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:08.338065 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:08.338066 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:08.338068 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:08.338152 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.338154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.339099 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.339100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.340364 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:08.340367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.341707 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.341709 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:08.341830 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.341836 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:33:08.342401 (+ 565us) client_negotiation.cc:240] Negotiation successful
0811 14:33:08.342470 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6781,"thread_start_us":117,"threads_started":1}
I20260811 14:33:08.343042 7240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c280f1decfc41b9881580d3209e38fd" candidate_uuid: "48a7836b306d4d18be80ce4ef90f2357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a6110c574874e22b02917ef6ad6a128"
I20260811 14:33:08.343144 7240 raft_consensus.cc:3037] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:08.343621 7087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c280f1decfc41b9881580d3209e38fd" candidate_uuid: "48a7836b306d4d18be80ce4ef90f2357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" is_pre_election: true
I20260811 14:33:08.343832 7087 raft_consensus.cc:2454] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 48a7836b306d4d18be80ce4ef90f2357 in term 0.
I20260811 14:33:08.343986 7086 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c280f1decfc41b9881580d3209e38fd" candidate_uuid: "48a7836b306d4d18be80ce4ef90f2357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84c9e2ebc1f14cb2860608f00cde4fec"
I20260811 14:33:08.344259 7086 raft_consensus.cc:3037] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:08.344866 7240 raft_consensus.cc:2454] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48a7836b306d4d18be80ce4ef90f2357 in term 1.
I20260811 14:33:08.345191 7319 leader_election.cc:302] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a6110c574874e22b02917ef6ad6a128, 48a7836b306d4d18be80ce4ef90f2357; no voters:
I20260811 14:33:08.345333 7492 raft_consensus.cc:2781] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:08.345602 7492 raft_consensus.cc:686] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 1 LEADER]: Becoming Leader. State: Replica: 48a7836b306d4d18be80ce4ef90f2357, State: Running, Role: LEADER
I20260811 14:33:08.345748 7086 raft_consensus.cc:2454] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 48a7836b306d4d18be80ce4ef90f2357 in term 1.
I20260811 14:33:08.345769 7492 consensus_queue.cc:237] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } }
I20260811 14:33:08.347985 6908 catalog_manager.cc:5495] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 reported cstate change: term changed from 0 to 1, leader changed from <none> to 48a7836b306d4d18be80ce4ef90f2357 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "48a7836b306d4d18be80ce4ef90f2357" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48a7836b306d4d18be80ce4ef90f2357" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 39049 } health_report { overall_health: HEALTHY } } }
W20260811 14:33:08.348623 7450 tablet.cc:2369] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:08.353113 7142 tablet.cc:2369] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:08.360903 7299 tablet.cc:2369] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:08.366842 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.363244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59638 (local address 127.23.186.195:39049)
0811 14:33:08.363292 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:08.363295 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:08.363304 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:08.363386 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:08.363387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:08.363410 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:08.363479 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:08.363683 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.364487 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.365650 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:08.365956 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:08.366035 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:08.366262 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:08.366431 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:33:08.366512 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:08.366726 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 14:33:08.366753 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:33:08.379069 7086 raft_consensus.cc:1261] T 8c280f1decfc41b9881580d3209e38fd P 84c9e2ebc1f14cb2860608f00cde4fec [term 1 FOLLOWER]: Refusing update from remote peer 48a7836b306d4d18be80ce4ef90f2357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:08.379485 7492 consensus_queue.cc:1035] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84c9e2ebc1f14cb2860608f00cde4fec" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 34131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:08.383337 7240 raft_consensus.cc:1261] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 1 FOLLOWER]: Refusing update from remote peer 48a7836b306d4d18be80ce4ef90f2357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:08.383867 7500 consensus_queue.cc:1035] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a6110c574874e22b02917ef6ad6a128" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:08.386880 7504 mvcc.cc:205] Tried to move back new op lower bound from 7317335197193216000 to 7317335197065306112. Current Snapshot: MvccSnapshot[applied={T|T < 7317335197193216000}]
I20260811 14:33:08.387045 7505 mvcc.cc:205] Tried to move back new op lower bound from 7317335197193216000 to 7317335197065306112. Current Snapshot: MvccSnapshot[applied={T|T < 7317335197193216000}]
I20260811 14:33:08.390578 6907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:08.390718 6907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:08.391975 6907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6939ab4097849079648b31af80da2ba: Sending DeleteTablet for 3 replicas of tablet 8c280f1decfc41b9881580d3209e38fd
I20260811 14:33:08.392506 7219 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c280f1decfc41b9881580d3209e38fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:08 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:55716
I20260811 14:33:08.392503 7064 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c280f1decfc41b9881580d3209e38fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:08 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:41088
I20260811 14:33:08.393092 7377 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c280f1decfc41b9881580d3209e38fd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:08 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:59630
I20260811 14:33:08.393762 7509 mvcc.cc:205] Tried to move back new op lower bound from 7317335197193216000 to 7317335197065306112. Current Snapshot: MvccSnapshot[applied={T|T < 7317335197193216000}]
I20260811 14:33:08.394229 7516 tablet_replica.cc:330] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: stopping tablet replica
I20260811 14:33:08.394420 7516 raft_consensus.cc:2229] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:08.394562 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 6950
I20260811 14:33:08.394698 7516 raft_consensus.cc:2258] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:08.401021 7516 ts_tablet_manager.cc:1905] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:08.402962 7517 tablet_replica.cc:330] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: stopping tablet replica
I20260811 14:33:08.403141 7517 raft_consensus.cc:2229] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:33:08.403273 7517 raft_consensus.cc:2258] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:08.405339 7517 ts_tablet_manager.cc:1905] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:08.407232 7516 ts_tablet_manager.cc:1918] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:08.407348 7516 log.cc:1196] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/8c280f1decfc41b9881580d3209e38fd
I20260811 14:33:08.407658 7516 ts_tablet_manager.cc:1939] T 8c280f1decfc41b9881580d3209e38fd P 2a6110c574874e22b02917ef6ad6a128: Deleting consensus metadata
I20260811 14:33:08.409374 6890 catalog_manager.cc:4841] TS 2a6110c574874e22b02917ef6ad6a128 (127.23.186.194:34463): tablet 8c280f1decfc41b9881580d3209e38fd (table test-table [id=50f74a1efe67473083381b6128a58974]) successfully deleted
W20260811 14:33:08.412879 6895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:34131 (error 108)
I20260811 14:33:08.412955 7517 ts_tablet_manager.cc:1918] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:08.413049 7517 log.cc:1196] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/8c280f1decfc41b9881580d3209e38fd
I20260811 14:33:08.413367 7517 ts_tablet_manager.cc:1939] T 8c280f1decfc41b9881580d3209e38fd P 48a7836b306d4d18be80ce4ef90f2357: Deleting consensus metadata
I20260811 14:33:08.413905 6891 catalog_manager.cc:4841] TS 48a7836b306d4d18be80ce4ef90f2357 (127.23.186.195:39049): tablet 8c280f1decfc41b9881580d3209e38fd (table test-table [id=50f74a1efe67473083381b6128a58974]) successfully deleted
I20260811 14:33:08.414556 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:08.414405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:34131 (local address 127.0.0.1:41100)
0811 14:33:08.414463 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:08.414512 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:34131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 14:33:08.414800 6895 catalog_manager.cc:4584] TS 84c9e2ebc1f14cb2860608f00cde4fec (127.23.186.193:34131): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c280f1decfc41b9881580d3209e38fd: Network error: Client connection negotiation failed: client connection to 127.23.186.193:34131: connect: Connection refused (error 111)
I20260811 14:33:08.415427 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 7145
I20260811 14:33:08.452330 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 7301
I20260811 14:33:08.462708 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 6869
I20260811 14:33:08.475016 24299 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 14:33:08.823060 24299 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 14:33:08.823179 24299 mini_ranger.cc:66] Stopping Ranger...
I20260811 14:33:09.180020 24299 mini_ranger.cc:68] Stopped Ranger
2026-08-11 14:33:09.180 UTC [5737] LOG: received smart shutdown request
2026-08-11 14:33:09.185 UTC [5737] LOG: background worker "logical replication launcher" (PID 5749) exited with exit code 1
2026-08-11 14:33:09.186 UTC [5741] LOG: shutting down
2026-08-11 14:33:09.441 UTC [5737] LOG: database system is shut down
2026-08-11T14:33:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (63700 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:33:09 dist-test-slave-cbsf krb5kdc[7530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:33:09 dist-test-slave-cbsf krb5kdc[7530](info): set up 2 sockets
Aug 11 14:33:09 dist-test-slave-cbsf krb5kdc[7530](info): commencing operation
krb5kdc: starting...
W20260811 14:33:10.843243 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:33:10 dist-test-slave-cbsf krb5kdc[7530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:33:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:33:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:33:11.021806 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39645
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39645,127.23.186.253:45997,127.23.186.252:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:11.118350 7545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:11.118623 7545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:11.118654 7545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:11.120040 7545 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:11.120211 7545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:11.120236 7545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:11.120251 7545 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:11.120266 7545 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:11.122169 7545 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39645,127.23.186.253:45997,127.23.186.252:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:11.122426 7545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:11.122632 7545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:11.128957 7550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:11.128957 7551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:11.128957 7554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:11.130618 7545 server_base.cc:1034] running on GCE node
I20260811 14:33:11.131363 7545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:11.132997 7545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:11.134313 7545 hybrid_clock.cc:648] HybridClock initialized: now 1786458791134276 us; error 74 us; skew 500 ppm
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:11.138883 7545 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:11.139647 7545 webserver.cc:460] Webserver started at http://127.23.186.254:37895/ using document root <none> and password file <none>
I20260811 14:33:11.139931 7545 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:11.140007 7545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:11.140214 7545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:11.141882 7545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "85e5d09e385bbcd0c7e2f0998cfce601"
server_key_iv: "716cd9a3bebe617a514aa1925e4a318a"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.142349 7545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "85e5d09e385bbcd0c7e2f0998cfce601"
server_key_iv: "716cd9a3bebe617a514aa1925e4a318a"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.144052 7545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:33:11.145162 7561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:11.145454 7545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:11.145592 7545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "85e5d09e385bbcd0c7e2f0998cfce601"
server_key_iv: "716cd9a3bebe617a514aa1925e4a318a"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.145685 7545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:11.165794 7545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:11.168598 7545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:11.168846 7545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:11.175019 7545 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:39645
I20260811 14:33:11.175103 7623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:39645 every 8 connection(s)
I20260811 14:33:11.175642 7545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:11.176962 7624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:11.179636 7624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.180480 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 7545
I20260811 14:33:11.180639 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:33:11.180908 24299 external_mini_cluster.cc:1428] Setting key afcffab4127196faedc8dab3a6d6cc2b
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.186.253@KRBTEST.COM; defaulting to no policy
I20260811 14:33:11.194676 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.180955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:39645 (local address 127.0.0.1:32856)
0811 14:33:11.181240 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.181296 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.182330 (+ 1034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.182810 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.182815 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.183144 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.183331 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.183336 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.184774 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.184776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.186055 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.186060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.186633 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.186634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.186731 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.187467 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.187481 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.189658 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.191974 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.191998 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.192015 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.192647 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.193112 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.193113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.193113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.193255 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.193729 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.193741 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.193918 (+ 177us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.194046 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.194382 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":146,"threads_started":1}
I20260811 14:33:11.194681 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.181009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.23.186.254:39645)
0811 14:33:11.181274 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.181287 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.181498 (+ 211us) server_negotiation.cc:407] Connection header received
0811 14:33:11.182592 (+ 1094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.182595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.182626 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.182737 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.183451 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.184633 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.186186 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.186529 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.186629 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.189794 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.189802 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.189803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.189812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.191783 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.192844 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.192848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.192849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.192969 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.193347 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.193348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.193348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.193637 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.193718 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.194151 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.194530 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":195,"server-negotiator.queue_time_us":229,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.253@KRBTEST.COM" created.
I20260811 14:33:11.196123 7624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.196810 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.196545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39908)
0811 14:33:11.196627 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.196689 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 14:33:11.196985 7564 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
I20260811 14:33:11.197590 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.197448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39914)
0811 14:33:11.197515 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.197547 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 14:33:11.197804 7624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:45997: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab.
Entry for principal kudu/127.23.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab.
I20260811 14:33:11.217924 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:45997
--webserver_interface=127.23.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab
--principal=kudu/127.23.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39645,127.23.186.253:45997,127.23.186.252:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:11.247164 7624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } attempt: 1
I20260811 14:33:11.247887 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.247726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39928)
0811 14:33:11.247802 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.247849 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 14:33:11.248492 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.248232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39940)
0811 14:33:11.248335 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.248421 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 14:33:11.248746 7624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:45997: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
I20260811 14:33:11.317018 7624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } attempt: 2
I20260811 14:33:11.317818 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.317579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39950)
0811 14:33:11.317687 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.317761 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 14:33:11.318421 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.318299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39954)
0811 14:33:11.318353 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.318391 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 14:33:11.318586 7624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:45997: Network error: Client connection negotiation failed: client connection to 127.23.186.253:45997: connect: Connection refused (error 111)
W20260811 14:33:11.323825 7633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:11.324383 7633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:11.324436 7633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:11.326664 7633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:11.326740 7633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:11.326761 7633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:11.326776 7633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:11.326792 7633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:11.329334 7633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39645,127.23.186.253:45997,127.23.186.252:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab
--principal=kudu/127.23.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:45997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:11.329757 7633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:11.330091 7633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:11.337227 7646 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:11.337221 7644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:11.337219 7643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:11.337515 7633 server_base.cc:1034] running on GCE node
I20260811 14:33:11.337898 7633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:11.338315 7633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:11.339537 7633 hybrid_clock.cc:648] HybridClock initialized: now 1786458791339519 us; error 48 us; skew 500 ppm
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:11.342458 7633 init.cc:377] Logged in from keytab as kudu/127.23.186.253@KRBTEST.COM (short username kudu)
I20260811 14:33:11.343024 7633 webserver.cc:460] Webserver started at http://127.23.186.253:33207/ using document root <none> and password file <none>
I20260811 14:33:11.343319 7633 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:11.343370 7633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:11.343520 7633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:11.344794 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data/instance:
uuid: "6f33d83bb51244b795ae8f0eab9f74c5"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "c9f50c4fa27c6d6de9ab2e7390353cb4"
server_key_iv: "d24092dbf44715b2fe8b96d7402e0652"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.345181 7633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal/instance:
uuid: "6f33d83bb51244b795ae8f0eab9f74c5"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "c9f50c4fa27c6d6de9ab2e7390353cb4"
server_key_iv: "d24092dbf44715b2fe8b96d7402e0652"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.346757 7633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:33:11.347743 7655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:11.347970 7633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:11.348086 7633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal
uuid: "6f33d83bb51244b795ae8f0eab9f74c5"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "c9f50c4fa27c6d6de9ab2e7390353cb4"
server_key_iv: "d24092dbf44715b2fe8b96d7402e0652"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.348160 7633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:11.373188 7633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:11.376636 7633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:11.376920 7633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:11.382742 7633 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.253:45997
I20260811 14:33:11.382839 7717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.253:45997 every 8 connection(s)
I20260811 14:33:11.383343 7633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
I20260811 14:33:11.384631 7718 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:11.386722 7718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.388180 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 7633
I20260811 14:33:11.388289 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-1/wal/instance
I20260811 14:33:11.388495 24299 external_mini_cluster.cc:1428] Setting key e3df266588564747c3810459ba1f169e
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.186.252@KRBTEST.COM; defaulting to no policy
I20260811 14:33:11.401734 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.387738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.23.186.254:39645)
0811 14:33:11.387835 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.387838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.388381 (+ 543us) server_negotiation.cc:407] Connection header received
0811 14:33:11.389313 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.389315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.389334 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.389439 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.390782 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.391588 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.393113 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.393444 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.393544 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.396820 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.396823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.396823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.396834 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.398766 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.400299 (+ 1533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.400302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.400302 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.400416 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.400833 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.400835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.400835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.401012 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.401094 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.401492 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.401633 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 14:33:11.402004 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.387654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:39645 (local address 127.0.0.1:32858)
0811 14:33:11.388066 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.388151 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.389153 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.389682 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.389688 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.390108 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.390642 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.390651 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.391770 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.391772 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.392963 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.392967 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.393562 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.393564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.393761 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.394499 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.394512 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.396515 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.398933 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.398956 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.398961 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.400008 (+ 1047us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.400508 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.400509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.400509 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.400678 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.401082 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.401092 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.401335 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.401458 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.401691 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":162,"threads_started":1}
I20260811 14:33:11.403249 7718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.252@KRBTEST.COM" created.
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:33:11.409782 7624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } attempt: 3
I20260811 14:33:11.414561 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.403697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39970)
0811 14:33:11.403810 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.403824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.403881 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.404532 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.404533 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.404765 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.404841 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.404844 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.405901 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.405902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.406739 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.406742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.407748 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.407750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.407816 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.408415 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.408421 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.410053 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.413328 (+ 3275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.413329 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.413330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.413544 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.413788 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.413788 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.413789 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.413852 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.414183 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.414184 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.414230 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.414305 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.414436 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 14:33:11.414703 7722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.403831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.23.186.253:45997)
0811 14:33:11.404275 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.404282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.404314 (+ 32us) server_negotiation.cc:407] Connection header received
0811 14:33:11.404339 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.404342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.404360 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.404444 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.404921 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.405705 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.406867 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.407638 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.407728 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.410171 (+ 2443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.410185 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.410185 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.410197 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.413236 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.413643 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.413647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.413647 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.413715 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.413918 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.413919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.413919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.414102 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.414211 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.414346 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.414541 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":289,"thread_start_us":164,"threads_started":1}
I20260811 14:33:11.416020 7718 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.416450 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.416312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48412)
0811 14:33:11.416366 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.416424 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:33:11.416659 7656 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
I20260811 14:33:11.417274 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.417168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48416)
0811 14:33:11.417216 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.417245 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
W20260811 14:33:11.417471 7718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:41725: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
I20260811 14:33:11.421937 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.410297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.23.186.253:45997)
0811 14:33:11.410502 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.410504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.410520 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:33:11.411148 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.411151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.411173 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.411308 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.411780 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.412961 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.414572 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.415028 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.415553 (+ 525us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.418405 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.418407 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.418407 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.418416 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.420265 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.420972 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.420973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.420974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.421037 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.421265 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.421266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.421266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.421560 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.421641 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.421766 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.421852 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":128,"threads_started":1}
I20260811 14:33:11.421949 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.410141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39976)
0811 14:33:11.410269 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.410282 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.410961 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.411331 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.411333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.411555 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.411659 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.411662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.413135 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.413137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.414408 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.414413 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.415150 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.415151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.415270 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.415868 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.415874 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.417948 (+ 2074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.420479 (+ 2531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.420480 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.420481 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.420832 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.421114 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.421115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.421116 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.421194 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.421634 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.421635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.421690 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.421748 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.421852 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 14:33:11.422422 7624 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.422927 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.422821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48430)
0811 14:33:11.422871 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.422905 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 14:33:11.423276 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.423186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48436)
0811 14:33:11.423230 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.423255 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:33:11.423416 7624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:41725: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab.
Entry for principal kudu/127.23.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab.
I20260811 14:33:11.429847 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:41725
--webserver_interface=127.23.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab
--principal=kudu/127.23.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:46607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39645,127.23.186.253:45997,127.23.186.252:41725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:11.454674 7624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } attempt: 1
I20260811 14:33:11.455507 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.455298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48440)
0811 14:33:11.455409 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.455464 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 14:33:11.456247 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.455961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48444)
0811 14:33:11.456113 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.456193 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260811 14:33:11.456494 7624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:41725: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
I20260811 14:33:11.466771 7718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } attempt: 1
I20260811 14:33:11.467365 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.467211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48446)
0811 14:33:11.467284 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.467332 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:33:11.467748 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.467649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48462)
0811 14:33:11.467695 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.467727 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 14:33:11.467834 7718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:41725: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
I20260811 14:33:11.531744 7624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } attempt: 2
I20260811 14:33:11.532541 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.532312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48474)
0811 14:33:11.532439 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.532490 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 14:33:11.533097 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.532972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48486)
0811 14:33:11.533036 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.533070 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 14:33:11.533206 7624 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:41725: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
I20260811 14:33:11.536033 7718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } attempt: 2
I20260811 14:33:11.536676 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.536520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48498)
0811 14:33:11.536595 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.536641 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 14:33:11.536585 7726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:11.536953 7726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:11.537034 7726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 14:33:11.537336 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.537151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48500)
0811 14:33:11.537255 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.537299 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 14:33:11.537496 7718 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:41725: Network error: Client connection negotiation failed: client connection to 127.23.186.252:41725: connect: Connection refused (error 111)
W20260811 14:33:11.538539 7726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:11.538625 7726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:11.538656 7726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:11.538686 7726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:11.538713 7726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:11.540436 7726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:46607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39645,127.23.186.253:45997,127.23.186.252:41725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab
--principal=kudu/127.23.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:41725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:11.540876 7726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:11.541181 7726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:11.547197 7736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:11.547166 7733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:11.547163 7731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:11.547911 7726 server_base.cc:1034] running on GCE node
I20260811 14:33:11.548218 7726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:11.548594 7726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:11.549764 7726 hybrid_clock.cc:648] HybridClock initialized: now 1786458791549733 us; error 57 us; skew 500 ppm
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:11.553717 7726 init.cc:377] Logged in from keytab as kudu/127.23.186.252@KRBTEST.COM (short username kudu)
I20260811 14:33:11.554425 7726 webserver.cc:460] Webserver started at http://127.23.186.252:41305/ using document root <none> and password file <none>
I20260811 14:33:11.554775 7726 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:11.554854 7726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:11.555027 7726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:11.556421 7726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data/instance:
uuid: "8063017c04b84074b6b4e9bd7c0ee919"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "6cbffff9aed527f01512493c6c7e29d0"
server_key_iv: "2a76504468c998b61fe0f6eb8be15fa3"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.557240 7726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal/instance:
uuid: "8063017c04b84074b6b4e9bd7c0ee919"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "6cbffff9aed527f01512493c6c7e29d0"
server_key_iv: "2a76504468c998b61fe0f6eb8be15fa3"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.559131 7726 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 14:33:11.560200 7744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:11.560518 7726 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:11.560673 7726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal
uuid: "8063017c04b84074b6b4e9bd7c0ee919"
format_stamp: "Formatted at 2026-08-11 14:33:11 on dist-test-slave-cbsf"
server_key: "6cbffff9aed527f01512493c6c7e29d0"
server_key_iv: "2a76504468c998b61fe0f6eb8be15fa3"
server_key_version: "encryptionkey@0"
I20260811 14:33:11.560838 7726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:11.582209 7726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:11.584548 7726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:11.584800 7726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:11.589634 7726 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.252:41725
I20260811 14:33:11.589775 7804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.252:41725 every 8 connection(s)
I20260811 14:33:11.590341 7726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
I20260811 14:33:11.591922 7805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:11.594046 7805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.599078 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 7726
I20260811 14:33:11.599277 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786458616937656-24299-0/minicluster-data/master-2/wal/instance
I20260811 14:33:11.599498 24299 external_mini_cluster.cc:1428] Setting key 4695d5d384ff0dda3f386316465403fa
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:33:11.606571 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.595044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.23.186.254:39645)
0811 14:33:11.595238 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.595241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.595466 (+ 225us) server_negotiation.cc:407] Connection header received
0811 14:33:11.596235 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.596237 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.596251 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.596321 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.597558 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.598190 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.599514 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.599776 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.599863 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.603104 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.603107 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.603107 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.603121 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.604823 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.605336 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.605338 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.605338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.605408 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.605802 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.605803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.605804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.605954 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.606032 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.606364 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.606469 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 14:33:11.606807 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.594885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:39645 (local address 127.0.0.1:32860)
0811 14:33:11.595250 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.595339 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.596096 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.596431 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.596437 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.596934 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.597247 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.597253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.598324 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.598326 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.599408 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.599411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.599879 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.599880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.600077 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.600890 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.600922 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.602980 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.604914 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.604924 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.604927 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.605247 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.605512 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.605514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.605514 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.605667 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.606034 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.606043 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.606221 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.606340 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.606528 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":151,"threads_started":1}
I20260811 14:33:11.607455 7805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.609488 7809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.600011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48516 (local address 127.23.186.252:41725)
0811 14:33:11.600310 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.600317 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.600359 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:33:11.600391 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.600393 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.600415 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.600510 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.601262 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.602205 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.603182 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.603515 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.603623 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.606148 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.606157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.606158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.606166 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.608103 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.608488 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.608492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.608493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.608555 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.608800 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.608801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.608801 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.608963 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.609145 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.609199 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.609351 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":248,"thread_start_us":172,"threads_started":1}
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:11.615446 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.607774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:45997 (local address 127.0.0.1:39978)
0811 14:33:11.607827 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.607837 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.607881 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.608090 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.608092 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.608240 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.608296 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.608297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.609140 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.609142 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.609873 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.609876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.610481 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.610482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.610560 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.611010 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.611015 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.612301 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.614107 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.614108 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.614109 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.614299 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.614592 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.614594 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.614595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.614687 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.615064 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.615066 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.615130 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.615273 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.615369 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":30}
I20260811 14:33:11.615500 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.607881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.23.186.253:45997)
0811 14:33:11.607952 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.607954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.607962 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:33:11.607992 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.607993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.608009 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.608084 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.608376 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.609055 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.609966 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.610339 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.610419 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.612400 (+ 1981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.612402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.612403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.612411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.613989 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.614375 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.614376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.614377 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.614439 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.614788 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.614790 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.614790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.614952 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.615040 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.615298 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.615400 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:33:11.616012 7805 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:33:11.618626 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.610484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32866 (local address 127.23.186.254:39645)
0811 14:33:11.610529 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.610531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.610582 (+ 51us) server_negotiation.cc:407] Connection header received
0811 14:33:11.610661 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.610662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.610672 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.610718 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.611225 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.611707 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.612443 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.612659 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.612781 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.615735 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.615736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.615737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.615745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.617417 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.617808 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.617809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.617810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.617869 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.618082 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.618083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.618083 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.618203 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.618399 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.618436 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.618516 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:33:11.623889 7809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.616444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48528 (local address 127.23.186.252:41725)
0811 14:33:11.616482 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.616484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.616490 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:33:11.616507 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.616507 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.616520 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.616589 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.616917 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.617419 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.618051 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.618267 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.618351 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.620563 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.620564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.620565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.620573 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.622517 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.623071 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.623072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.623072 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.623132 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.623351 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.623352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.623352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.623501 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.623556 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.623723 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.623814 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":22}
I20260811 14:33:11.624009 7807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.616320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48528)
0811 14:33:11.616360 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.616373 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.616414 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.616591 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.616593 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.616808 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.616843 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.616844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.617499 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.617501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.617973 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.617974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.618349 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.618350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.618398 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.618805 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.618810 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.620421 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.622732 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.622733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.622733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.622978 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.623200 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.623201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.623202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.623278 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.623565 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.623565 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.623624 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.623704 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.623894 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21,"mutex_wait_us":68}
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458790, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:33:11.626237 7805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: Bootstrap starting.
I20260811 14:33:11.627547 7805 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:11.627938 7805 log.cc:824] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:11.628625 7718 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } attempt: 3
I20260811 14:33:11.628794 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.619299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.23.186.253:45997)
0811 14:33:11.619342 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.619344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.619504 (+ 160us) server_negotiation.cc:407] Connection header received
0811 14:33:11.619560 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.619562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.619581 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.619680 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.620473 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.621060 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.621802 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.622175 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.622276 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.625161 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.625163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.625163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.625175 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.627051 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.627482 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.627484 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.627484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.627557 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.627826 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.627827 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.627827 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.628009 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.628186 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.628442 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.628543 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:33:11.629561 7805 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: No bootstrap required, opened a new log
I20260811 14:33:11.630646 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:33:11.631767 7805 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.632119 7805 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:11.632185 7805 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8063017c04b84074b6b4e9bd7c0ee919, State: Initialized, Role: FOLLOWER
I20260811 14:33:11.632328 7805 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.632333 7624 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } attempt: 3
I20260811 14:33:11.632762 7805 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:11.632803 7816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.632907 7816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
W20260811 14:33:11.634913 7831 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:33:11.634967 7831 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:33:11.638860 7720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.629011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48534)
0811 14:33:11.629067 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.629079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.629124 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.629392 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.629394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.629580 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.629645 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.629647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.630603 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.630604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.631427 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.631430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.632097 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.632098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.632169 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.633299 (+ 1130us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.633307 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.635143 (+ 1836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.637508 (+ 2365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.637509 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.637510 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.637797 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.638057 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.638058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.638059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.638131 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.638501 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.638502 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.638556 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.638627 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.638744 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:33:11.639205 7809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.629060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48534 (local address 127.23.186.252:41725)
0811 14:33:11.629104 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.629106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.629116 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:33:11.629181 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.629182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.629194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.629255 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.629772 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.630518 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.631531 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.631877 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.632101 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.635341 (+ 3240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.635343 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.635343 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.635351 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.637389 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.637893 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.637895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.637895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.637976 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.638204 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.638205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.638205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.638391 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.638468 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.639004 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.639103 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:33:11.640985 7718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5: Bootstrap starting.
I20260811 14:33:11.637970 7726 master_runner.cc:186] Error getting master registration for 127.23.186.254:39645: OK, instance_id { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" instance_seqno: 1786458791173145 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:33:11.641423 7726 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:33:11.641889 7718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:11.642222 7718 log.cc:824] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:11.646291 7627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.633571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.23.186.254:39645)
0811 14:33:11.633623 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.633625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.634425 (+ 800us) server_negotiation.cc:407] Connection header received
0811 14:33:11.634506 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.634507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.634522 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.634586 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.636826 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.637941 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.639157 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.639500 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.639555 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.641787 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.641790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.641791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.641807 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.644216 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.644737 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.644738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.644739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.644805 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.645542 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.645544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.645544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.645716 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.645810 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.646056 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.646168 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 11 14:33:11 dist-test-slave-cbsf krb5kdc[7530](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458791, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:33:11.650473 7718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5: No bootstrap required, opened a new log
I20260811 14:33:11.652275 7626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.632989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:41725 (local address 127.0.0.1:48560)
0811 14:33:11.633037 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:11.633051 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:11.633099 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:11.642018 (+ 8919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:11.642020 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:11.642228 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:11.642283 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.642285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.643239 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.643241 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.644011 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:11.644014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.644533 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.644534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:11.644598 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.645191 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:11.645196 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:11.647324 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:11.650781 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.650783 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.650784 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.651047 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.651496 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:11.651497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:11.651498 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:11.651579 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:11.651884 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:11.651885 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:11.651941 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 14:33:11.652027 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 14:33:11.652168 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 14:33:11.652297 7819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.633131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48560 (local address 127.23.186.252:41725)
0811 14:33:11.635112 (+ 1981us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.635114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.635123 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:11.635144 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.635145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.635154 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.635272 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.642544 (+ 7272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.643134 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.644123 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.644429 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.644859 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.647455 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.647457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.647458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.647469 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.650683 (+ 3214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.651202 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.651204 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.651205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.651387 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.651656 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.651657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.651657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.651804 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.651886 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.652058 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.652171 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1958,"thread_start_us":91,"threads_started":1}
I20260811 14:33:11.652935 7718 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.653092 7718 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:11.653139 7718 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f33d83bb51244b795ae8f0eab9f74c5, State: Initialized, Role: FOLLOWER
I20260811 14:33:11.653287 7718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.653649 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.653755 7718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:11.654364 7624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: Bootstrap starting.
I20260811 14:33:11.655184 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.632798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.23.186.252:41725)
0811 14:33:11.633304 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.633307 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.634603 (+ 1296us) server_negotiation.cc:407] Connection header received
0811 14:33:11.634708 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.634709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.634722 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.634772 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.639995 (+ 5223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.641158 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.642446 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.642847 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.642903 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.647627 (+ 4724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.647629 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.647629 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.647637 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.653590 (+ 5953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.654146 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.654147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.654148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.654245 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.654479 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.654480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.654481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.654627 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.654714 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.654955 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.655073 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":89,"threads_started":1}
I20260811 14:33:11.655370 7624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:11.655762 7624 log.cc:824] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:11.656002 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:33:11.656857 7850 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:33:11.656911 7850 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:33:11.656978 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:11.634166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39986 (local address 127.23.186.253:45997)
0811 14:33:11.634214 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:11.634217 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:11.643505 (+ 9288us) server_negotiation.cc:407] Connection header received
0811 14:33:11.643643 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:11.643645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:11.643689 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:11.643724 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:11.645771 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.646485 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.647619 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:11.647973 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:11.648062 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:11.649664 (+ 1602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:11.649667 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:11.649667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:11.649676 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:11.651839 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.655907 (+ 4068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.655909 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.655910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.655985 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:11.656237 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:11.656238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:11.656238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:11.656394 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:11.656485 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:11.656766 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 14:33:11.656868 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:33:11.656788 7633 master_runner.cc:186] Error getting master registration for 127.23.186.254:39645: OK, instance_id { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" instance_seqno: 1786458791173145 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:33:11.659403 7633 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:33:11.663743 7624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: No bootstrap required, opened a new log
I20260811 14:33:11.665920 7624 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.666046 7624 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:11.666069 7624 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccb94a0c7b3140bab2644b18c6f3b5ee, State: Initialized, Role: FOLLOWER
I20260811 14:33:11.666213 7624 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.666501 7855 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.666585 7624 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:11.666572 7855 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:33:11.670148 7568 tablet.cc:2369] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:11.671348 7545 master_runner.cc:263] No leader master found from master ccb94a0c7b3140bab2644b18c6f3b5ee
I20260811 14:33:11.671404 7545 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 14:33:11.672034 7868 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:33:11.672179 7868 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:33:11.816530 7835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:11.816740 7835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.817065 7835 leader_election.cc:288] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ccb94a0c7b3140bab2644b18c6f3b5ee (127.23.186.254:39645), 8063017c04b84074b6b4e9bd7c0ee919 (127.23.186.252:41725)
I20260811 14:33:11.817672 7772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8063017c04b84074b6b4e9bd7c0ee919" is_pre_election: true
I20260811 14:33:11.817862 7772 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f33d83bb51244b795ae8f0eab9f74c5 in term 0.
I20260811 14:33:11.818202 7593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" is_pre_election: true
I20260811 14:33:11.818337 7593 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f33d83bb51244b795ae8f0eab9f74c5 in term 0.
I20260811 14:33:11.818572 7657 leader_election.cc:302] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f33d83bb51244b795ae8f0eab9f74c5, ccb94a0c7b3140bab2644b18c6f3b5ee; no voters:
I20260811 14:33:11.819082 7835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:33:11.819152 7835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:33:11.819176 7835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:11.820240 7835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.820391 7835 leader_election.cc:288] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [CANDIDATE]: Term 1 election: Requested vote from peers ccb94a0c7b3140bab2644b18c6f3b5ee (127.23.186.254:39645), 8063017c04b84074b6b4e9bd7c0ee919 (127.23.186.252:41725)
I20260811 14:33:11.821000 7593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee"
I20260811 14:33:11.821116 7593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:11.822103 7593 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f33d83bb51244b795ae8f0eab9f74c5 in term 1.
I20260811 14:33:11.822391 7657 leader_election.cc:302] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f33d83bb51244b795ae8f0eab9f74c5, ccb94a0c7b3140bab2644b18c6f3b5ee; no voters:
I20260811 14:33:11.822541 7772 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8063017c04b84074b6b4e9bd7c0ee919"
I20260811 14:33:11.822650 7772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:11.823619 7772 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f33d83bb51244b795ae8f0eab9f74c5 in term 1.
I20260811 14:33:11.823980 7835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:11.824237 7835 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [term 1 LEADER]: Becoming Leader. State: Replica: 6f33d83bb51244b795ae8f0eab9f74c5, State: Running, Role: LEADER
I20260811 14:33:11.824383 7835 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } }
I20260811 14:33:11.824904 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f33d83bb51244b795ae8f0eab9f74c5. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.824985 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:11.826462 7872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:11.826751 7872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:11.829286 7772 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [term 1 FOLLOWER]: Refusing update from remote peer 6f33d83bb51244b795ae8f0eab9f74c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:11.829758 7835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:11.832335 7816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f33d83bb51244b795ae8f0eab9f74c5. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.832511 7816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:33:11.833230 7593 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [term 1 FOLLOWER]: Refusing update from remote peer 6f33d83bb51244b795ae8f0eab9f74c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:11.834115 7835 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:11.834779 7835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.834898 7835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:11.835261 7816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.835379 7816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:33:11.835832 7873 mvcc.cc:205] Tried to move back new op lower bound from 7317335211329265664 to 7317335211313364992. Current Snapshot: MvccSnapshot[applied={T|T < 7317335211329265664}]
I20260811 14:33:11.836720 7855 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f33d83bb51244b795ae8f0eab9f74c5. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.836834 7855 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:33:11.836890 7855 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.836935 7855 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:33:11.837067 7872 catalog_manager.cc:1347] Generated new cluster ID: 0776958b67864613826aeb977a6bbbce
I20260811 14:33:11.837116 7872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:11.837355 7874 mvcc.cc:205] Tried to move back new op lower bound from 7317335211329265664 to 7317335211313364992. Current Snapshot: MvccSnapshot[applied={T|T < 7317335211329265664 or (T in {7317335211329265664})}]
I20260811 14:33:11.841701 7879 mvcc.cc:205] Tried to move back new op lower bound from 7317335211329265664 to 7317335211313364992. Current Snapshot: MvccSnapshot[applied={T|T < 7317335211329265664}]
I20260811 14:33:11.859047 7870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ccb94a0c7b3140bab2644b18c6f3b5ee" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39645 } } peers { permanent_uuid: "6f33d83bb51244b795ae8f0eab9f74c5" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 45997 } } peers { permanent_uuid: "8063017c04b84074b6b4e9bd7c0ee919" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 41725 } } }
I20260811 14:33:11.859217 7870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:11.860893 7872 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:33:11.861545 7872 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:11.875469 7872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f33d83bb51244b795ae8f0eab9f74c5: Generated new TSK 0
I20260811 14:33:11.875869 7872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:33:12.635744 7831 catalog_manager.cc:1259] Loaded cluster ID: 0776958b67864613826aeb977a6bbbce
I20260811 14:33:12.635864 7831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: loading cluster ID for follower catalog manager: success
I20260811 14:33:12.636837 7831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: acquiring CA information for follower catalog manager: success
I20260811 14:33:12.637181 7831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8063017c04b84074b6b4e9bd7c0ee919: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:33:12.672802 7868 catalog_manager.cc:1259] Loaded cluster ID: 0776958b67864613826aeb977a6bbbce
I20260811 14:33:12.672940 7868 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: loading cluster ID for follower catalog manager: success
I20260811 14:33:12.673868 7868 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: acquiring CA information for follower catalog manager: success
I20260811 14:33:12.674232 7868 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ccb94a0c7b3140bab2644b18c6f3b5ee: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:33:12.945686 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 7545
I20260811 14:33:12.958359 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 7633
I20260811 14:33:12.969071 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 7726
2026-08-11T14:33:12Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3399 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:33:13 dist-test-slave-cbsf krb5kdc[7888](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:33:13 dist-test-slave-cbsf krb5kdc[7888](info): set up 2 sockets
Aug 11 14:33:13 dist-test-slave-cbsf krb5kdc[7888](info): commencing operation
krb5kdc: starting...
W20260811 14:33:14.183547 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:33:14 dist-test-slave-cbsf krb5kdc[7888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:33:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:33:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:33:14.341727 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33661
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:43807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:33661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:14.454113 7903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:14.454504 7903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:14.454540 7903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:14.456147 7903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:14.456233 7903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:14.456250 7903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:14.456265 7903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:14.456278 7903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:14.458564 7903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:43807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:33661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:14.458997 7903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:14.459312 7903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:14.466389 7911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:14.466389 7909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:14.466389 7908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:14.467000 7903 server_base.cc:1034] running on GCE node
I20260811 14:33:14.467410 7903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:14.467841 7903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:14.469065 7903 hybrid_clock.cc:648] HybridClock initialized: now 1786458794469038 us; error 50 us; skew 500 ppm
Aug 11 14:33:14 dist-test-slave-cbsf krb5kdc[7888](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458794, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:14.479619 7903 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:14.480495 7903 webserver.cc:460] Webserver started at http://127.23.186.254:46029/ using document root <none> and password file <none>
I20260811 14:33:14.480938 7903 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:14.481033 7903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:14.481251 7903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:14.482826 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "5fc57be18f4b48cca1aabeef04d7c8b2"
format_stamp: "Formatted at 2026-08-11 14:33:14 on dist-test-slave-cbsf"
server_key: "761d502da33421f836951e7234f5d932"
server_key_iv: "26e737265e7d9cd42a7079225f3857bf"
server_key_version: "encryptionkey@0"
I20260811 14:33:14.483342 7903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "5fc57be18f4b48cca1aabeef04d7c8b2"
format_stamp: "Formatted at 2026-08-11 14:33:14 on dist-test-slave-cbsf"
server_key: "761d502da33421f836951e7234f5d932"
server_key_iv: "26e737265e7d9cd42a7079225f3857bf"
server_key_version: "encryptionkey@0"
I20260811 14:33:14.485466 7903 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 14:33:14.486790 7919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:14.487231 7903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:14.487455 7903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "5fc57be18f4b48cca1aabeef04d7c8b2"
format_stamp: "Formatted at 2026-08-11 14:33:14 on dist-test-slave-cbsf"
server_key: "761d502da33421f836951e7234f5d932"
server_key_iv: "26e737265e7d9cd42a7079225f3857bf"
server_key_version: "encryptionkey@0"
I20260811 14:33:14.487593 7903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:14.507622 7903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:14.511042 7903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:14.511296 7903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:14.517912 7903 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:33661
I20260811 14:33:14.518107 7986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:33661 every 8 connection(s)
I20260811 14:33:14.518520 7903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:14.519724 7987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:14.522855 7987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2: Bootstrap starting.
I20260811 14:33:14.522948 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 7903
I20260811 14:33:14.523051 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:33:14.523247 24299 external_mini_cluster.cc:1428] Setting key 5c377a07891e0bd21cbf34581edff318
I20260811 14:33:14.523819 7987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:14.524273 7987 log.cc:824] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:14.525272 7987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2: No bootstrap required, opened a new log
I20260811 14:33:14.527733 7987 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33661 } }
I20260811 14:33:14.527981 7987 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:14.528026 7987 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fc57be18f4b48cca1aabeef04d7c8b2, State: Initialized, Role: FOLLOWER
I20260811 14:33:14.528179 7987 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33661 } }
I20260811 14:33:14.528264 7987 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:14.528296 7987 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:14.528343 7987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:14.529234 7987 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33661 } }
I20260811 14:33:14.529378 7987 leader_election.cc:302] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5fc57be18f4b48cca1aabeef04d7c8b2; no voters:
I20260811 14:33:14.529632 7987 leader_election.cc:288] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:33:14.529654 7994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:14.529888 7994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [term 1 LEADER]: Becoming Leader. State: Replica: 5fc57be18f4b48cca1aabeef04d7c8b2, State: Running, Role: LEADER
I20260811 14:33:14.529963 7987 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:14.530017 7994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33661 } }
I20260811 14:33:14.531762 7995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33661 } } }
I20260811 14:33:14.532159 7995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [sys.catalog]: This master's current role is: LEADER
Aug 11 14:33:14 dist-test-slave-cbsf krb5kdc[7888](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:14.532006 7996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fc57be18f4b48cca1aabeef04d7c8b2. Latest consensus state: current_term: 1 leader_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5fc57be18f4b48cca1aabeef04d7c8b2" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33661 } } }
I20260811 14:33:14.532344 7996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2 [sys.catalog]: This master's current role is: LEADER
W20260811 14:33:14.532887 8010 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:33:14.533044 8010 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:33:14.533224 8011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:14.533495 8011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:14.536065 8011 catalog_manager.cc:1347] Generated new cluster ID: 02677fd501294e56bc68fb25fc50e01d
I20260811 14:33:14.536149 8011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:14.538414 7991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:14.523937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34382 (local address 127.23.186.254:33661)
0811 14:33:14.524156 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:14.524171 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:14.524204 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:33:14.524996 (+ 792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:14.525006 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:14.525070 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:14.525214 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:14.526251 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:14.527443 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:14.528796 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:14.529305 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:14.529453 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:14.533166 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:14.533171 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:14.533171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:14.533188 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:14.535665 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:14.536318 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:14.536324 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:14.536324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:14.536485 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:14.536756 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:14.536757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:14.536757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:14.537190 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:14.537351 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:14.537952 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 14:33:14.538143 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":107,"threads_started":1}
I20260811 14:33:14.540033 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:33:14.545281 8011 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:33:14.546016 8011 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:14.550479 8011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5fc57be18f4b48cca1aabeef04d7c8b2: Generated new TSK 0
I20260811 14:33:14.550884 8011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:33:14.595396 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 7903
2026-08-11T14:33:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1626 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:33:14 dist-test-slave-cbsf krb5kdc[8019](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:33:14 dist-test-slave-cbsf krb5kdc[8019](info): set up 2 sockets
Aug 11 14:33:14 dist-test-slave-cbsf krb5kdc[8019](info): commencing operation
krb5kdc: starting...
W20260811 14:33:15.834096 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:33:15 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:33:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:33:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:33:16.005645 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35117
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:35117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:16.113281 8034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:16.113642 8034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:16.113678 8034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:16.115839 8034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:16.115936 8034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:16.115953 8034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:16.115967 8034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:16.115980 8034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:16.118530 8034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:35117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:16.119009 8034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:16.119370 8034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:16.125995 8045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.125995 8048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.126024 8043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:16.126365 8034 server_base.cc:1034] running on GCE node
I20260811 14:33:16.126741 8034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:16.127171 8034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:16.128399 8034 hybrid_clock.cc:648] HybridClock initialized: now 1786458796128372 us; error 47 us; skew 500 ppm
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:16.131830 8034 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:16.132440 8034 webserver.cc:460] Webserver started at http://127.23.186.254:34437/ using document root <none> and password file <none>
I20260811 14:33:16.132808 8034 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:16.132889 8034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:16.133093 8034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:16.134400 8034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "cea0c0279558468aae45096a18cfe1a4"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "359382dcf95c40928dfb282da4fdea03"
server_key_iv: "13aae4bd9cae2637d7a74124cf6b07ea"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.134833 8034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "cea0c0279558468aae45096a18cfe1a4"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "359382dcf95c40928dfb282da4fdea03"
server_key_iv: "13aae4bd9cae2637d7a74124cf6b07ea"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.136879 8034 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 14:33:16.138717 8058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.139294 8034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:33:16.139421 8034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "cea0c0279558468aae45096a18cfe1a4"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "359382dcf95c40928dfb282da4fdea03"
server_key_iv: "13aae4bd9cae2637d7a74124cf6b07ea"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.139583 8034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:16.177765 8034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:16.181511 8034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:16.181689 8034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:16.187456 8034 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:35117
I20260811 14:33:16.187505 8121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:35117 every 8 connection(s)
I20260811 14:33:16.187952 8034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:16.189419 8122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:16.192315 8122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap starting.
I20260811 14:33:16.193192 8122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:16.193539 8122 log.cc:824] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:16.195124 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8034
I20260811 14:33:16.195187 8122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: No bootstrap required, opened a new log
I20260811 14:33:16.195284 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:33:16.195508 24299 external_mini_cluster.cc:1428] Setting key 1fb9a8f6d3766ab8a7d102078ed7c029
I20260811 14:33:16.198382 8122 raft_consensus.cc:348] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:16.198702 8122 raft_consensus.cc:374] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:16.198753 8122 raft_consensus.cc:729] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0c0279558468aae45096a18cfe1a4, State: Initialized, Role: FOLLOWER
I20260811 14:33:16.198941 8122 consensus_queue.cc:260] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:16.199057 8122 raft_consensus.cc:388] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:16.199107 8122 raft_consensus.cc:482] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:16.199160 8122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:16.200829 8122 raft_consensus.cc:504] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:16.200985 8122 leader_election.cc:302] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cea0c0279558468aae45096a18cfe1a4; no voters:
I20260811 14:33:16.201339 8122 leader_election.cc:288] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:33:16.201596 8130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:16.201777 8122 sys_catalog.cc:564] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:16.201956 8130 raft_consensus.cc:686] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 LEADER]: Becoming Leader. State: Replica: cea0c0279558468aae45096a18cfe1a4, State: Running, Role: LEADER
I20260811 14:33:16.202198 8130 consensus_queue.cc:237] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:16.203351 8132 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cea0c0279558468aae45096a18cfe1a4. Latest consensus state: current_term: 1 leader_uuid: "cea0c0279558468aae45096a18cfe1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } } }
I20260811 14:33:16.203500 8132 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:16.204056 8131 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cea0c0279558468aae45096a18cfe1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } } }
I20260811 14:33:16.204172 8131 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: This master's current role is: LEADER
W20260811 14:33:16.204595 8149 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:33:16.204670 8149 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458795, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:16.206854 8147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:16.207304 8147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:16.209613 8147 catalog_manager.cc:1347] Generated new cluster ID: 07157ac69f5d4e2c8353e9dcd0dfe016
I20260811 14:33:16.209686 8147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:16.213284 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.196442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.23.186.254:35117)
0811 14:33:16.196749 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.196762 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.196832 (+ 70us) server_negotiation.cc:407] Connection header received
0811 14:33:16.198321 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.198335 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.198427 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.198627 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:16.199757 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.201090 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.202437 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.203279 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.203449 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.207813 (+ 4364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:16.207820 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:16.207821 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:16.207841 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:16.210390 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.210931 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.210937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.210938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.211107 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.211349 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.211350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.211350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.211965 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:16.212588 (+ 623us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.212766 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.213018 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":153,"threads_started":1}
I20260811 14:33:16.229151 8147 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:33:16.229797 8147 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:16.237356 8147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Generated new TSK 0
I20260811 14:33:16.237658 8147 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:33:16.277737 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:35117
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:16.388206 8157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:16.388500 8157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:16.388533 8157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:16.390072 8157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:16.390169 8157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:16.390204 8157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:16.392735 8157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:16.393118 8157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:16.393428 8157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:33:16.403182 8157 server_base.cc:1034] running on GCE node
W20260811 14:33:16.403496 8162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.403182 8165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.403187 8163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:16.403999 8157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:16.404345 8157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:16.405611 8157 hybrid_clock.cc:648] HybridClock initialized: now 1786458796405578 us; error 48 us; skew 500 ppm
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:16.409150 8157 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:16.409855 8157 webserver.cc:460] Webserver started at http://127.23.186.193:34827/ using document root <none> and password file <none>
I20260811 14:33:16.410176 8157 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:16.410256 8157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:16.410418 8157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:16.411708 8157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "7473880090e544dda868033a4ce5370a"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "c0c9bdc4c166eefa726dc4a1fc3c3568"
server_key_iv: "ef61061aa4fea791888b7cd4e51f8f38"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.412178 8157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "7473880090e544dda868033a4ce5370a"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "c0c9bdc4c166eefa726dc4a1fc3c3568"
server_key_iv: "ef61061aa4fea791888b7cd4e51f8f38"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.414654 8157 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 14:33:16.415767 8174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.416079 8157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:16.416240 8157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "7473880090e544dda868033a4ce5370a"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "c0c9bdc4c166eefa726dc4a1fc3c3568"
server_key_iv: "ef61061aa4fea791888b7cd4e51f8f38"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.416352 8157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:16.434053 8157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:16.437139 8157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:16.437362 8157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:16.437737 8157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:16.438198 8157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:16.438264 8157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.438318 8157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:16.438349 8157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.446043 8157 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:46117
I20260811 14:33:16.446110 8306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:46117 every 8 connection(s)
I20260811 14:33:16.446537 8157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:16.448225 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8157
I20260811 14:33:16.448336 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:33:16.448546 24299 external_mini_cluster.cc:1428] Setting key eae397eeeb4cc4d05847ee8bd6161f42
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
I20260811 14:33:16.460539 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.447751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:42735 (local address 127.23.186.254:35117)
0811 14:33:16.447851 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.447854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.448056 (+ 202us) server_negotiation.cc:407] Connection header received
0811 14:33:16.448633 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.448635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.448659 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.448795 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:16.450330 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.450908 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.452472 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.452797 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.452894 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.455982 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:16.455986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:16.455986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:16.456006 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:16.458469 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.459204 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.459206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.459206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.459293 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.459655 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.459656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.459656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.459822 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:16.459931 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.460297 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.460423 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 14:33:16.460852 8310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.447626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:42735)
0811 14:33:16.447815 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:16.447873 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:16.448509 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:16.448910 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:16.448917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:16.449797 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:16.450161 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.450170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.451036 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.451037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.452335 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.452339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.452916 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.452918 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.453092 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.453759 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:16.453770 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:16.455805 (+ 2035us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:16.458654 (+ 2849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:16.458673 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:16.458676 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:16.459048 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:16.459403 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:16.459404 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:16.459405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:16.459555 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:16.459935 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:16.459945 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:16.460123 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 14:33:16.460262 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 14:33:16.460495 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260811 14:33:16.461622 8308 heartbeater.cc:347] Connected to a master server at 127.23.186.254:35117
I20260811 14:33:16.461937 8308 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:16.462165 8308 heartbeater.cc:511] Master 127.23.186.254:35117 requested a full tablet report, sending...
I20260811 14:33:16.462791 8078 ts_manager.cc:194] Registered new tserver with Master: 7473880090e544dda868033a4ce5370a (127.23.186.193:46117)
I20260811 14:33:16.463536 8078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:42735
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:33:16.490015 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:35117
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:16.599218 8313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:16.599601 8313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:16.599663 8313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:16.601466 8313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:16.601575 8313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:16.601634 8313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:16.605140 8313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:16.605567 8313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:16.605866 8313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:16.612406 8319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.612411 8320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.612406 8323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:16.612699 8313 server_base.cc:1034] running on GCE node
I20260811 14:33:16.613003 8313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:16.613423 8313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:16.614697 8313 hybrid_clock.cc:648] HybridClock initialized: now 1786458796614667 us; error 70 us; skew 500 ppm
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:16.618927 8313 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:16.619685 8313 webserver.cc:460] Webserver started at http://127.23.186.194:40091/ using document root <none> and password file <none>
I20260811 14:33:16.620031 8313 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:16.620095 8313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:16.620265 8313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:16.621634 8313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "c0543a258afc4b59b7a2be4b0d950d94"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "2fc8f087f4e16e514266c9380a10a664"
server_key_iv: "9585807f375cb67009a6e08af3a8590e"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.622051 8313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "c0543a258afc4b59b7a2be4b0d950d94"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "2fc8f087f4e16e514266c9380a10a664"
server_key_iv: "9585807f375cb67009a6e08af3a8590e"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.623641 8313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:33:16.624614 8330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.624859 8313 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:16.624964 8313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "c0543a258afc4b59b7a2be4b0d950d94"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "2fc8f087f4e16e514266c9380a10a664"
server_key_iv: "9585807f375cb67009a6e08af3a8590e"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.625046 8313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:16.634577 8313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:16.638382 8313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:16.638646 8313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:16.639035 8313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:16.639598 8313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:16.639673 8313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.639719 8313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:16.639772 8313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.648388 8313 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33789
I20260811 14:33:16.648440 8465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33789 every 8 connection(s)
I20260811 14:33:16.648955 8313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:33:16.650512 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8313
I20260811 14:33:16.650755 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:33:16.650980 24299 external_mini_cluster.cc:1428] Setting key 05e2daaddecb447b684ce312203a8c4e
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:16.664392 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.649908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:57031 (local address 127.23.186.254:35117)
0811 14:33:16.650037 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.650039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.650258 (+ 219us) server_negotiation.cc:407] Connection header received
0811 14:33:16.650876 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.650878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.650895 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.650981 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:16.652763 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.653651 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.655637 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.655918 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.656006 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.659918 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:16.659941 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:16.659942 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:16.659957 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:16.662251 (+ 2294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.663010 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.663012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.663013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.663123 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.663469 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.663470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.663471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.663653 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:16.663749 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.664128 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.664254 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 14:33:16.664731 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.649860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:57031)
0811 14:33:16.650092 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:16.650145 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:16.650764 (+ 619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:16.651076 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:16.651081 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:16.652233 (+ 1152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:16.652483 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.652490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.653877 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.653879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.655402 (+ 1523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.655410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.656054 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.656056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.656293 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.657247 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:16.657264 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:16.659638 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:16.662443 (+ 2805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:16.662474 (+ 31us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:16.662478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:16.662891 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:16.663214 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:16.663216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:16.663216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:16.663391 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:16.663727 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:16.663737 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:16.663958 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 14:33:16.664097 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 14:33:16.664366 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":127,"threads_started":1}
I20260811 14:33:16.665807 8466 heartbeater.cc:347] Connected to a master server at 127.23.186.254:35117
I20260811 14:33:16.666014 8466 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:16.666374 8466 heartbeater.cc:511] Master 127.23.186.254:35117 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
I20260811 14:33:16.667063 8078 ts_manager.cc:194] Registered new tserver with Master: c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
I20260811 14:33:16.667567 8078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:57031
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:33:16.693933 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:35117
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:16.797654 8471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:16.797912 8471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:16.797950 8471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:16.799435 8471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:16.799513 8471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:16.799548 8471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:16.801608 8471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:16.801970 8471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:16.802261 8471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:16.808938 8477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.808892 8480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:16.808897 8478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:16.809329 8471 server_base.cc:1034] running on GCE node
I20260811 14:33:16.809665 8471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:16.810029 8471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:16.811379 8471 hybrid_clock.cc:648] HybridClock initialized: now 1786458796811348 us; error 33 us; skew 500 ppm
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:16.815829 8471 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:16.816603 8471 webserver.cc:460] Webserver started at http://127.23.186.195:35551/ using document root <none> and password file <none>
I20260811 14:33:16.816967 8471 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:16.817051 8471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:16.817251 8471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:16.818517 8471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "a88030d578454489985b6252c4dab642"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "67f84164bcd567be5bd713617b9c93c1"
server_key_iv: "5352614b9bd5a29aaa2cecbe0bb273a1"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.818943 8471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "a88030d578454489985b6252c4dab642"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "67f84164bcd567be5bd713617b9c93c1"
server_key_iv: "5352614b9bd5a29aaa2cecbe0bb273a1"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.820632 8471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:33:16.821883 8487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.822464 8471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:16.822619 8471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "a88030d578454489985b6252c4dab642"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "67f84164bcd567be5bd713617b9c93c1"
server_key_iv: "5352614b9bd5a29aaa2cecbe0bb273a1"
server_key_version: "encryptionkey@0"
I20260811 14:33:16.822691 8471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:16.840158 8471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:16.843611 8471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:16.843761 8471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:16.844071 8471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:16.844537 8471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:16.844614 8471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.844666 8471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:16.844709 8471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:16.851491 8471 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:34849
I20260811 14:33:16.851547 8619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:34849 every 8 connection(s)
I20260811 14:33:16.851976 8471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:33:16.852598 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8471
I20260811 14:33:16.852797 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:33:16.853045 24299 external_mini_cluster.cc:1428] Setting key 4dd26b4e96ff4d9471fd394b51b6b9eb
Aug 11 14:33:16 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458796, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:16.866542 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.853134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:41293 (local address 127.23.186.254:35117)
0811 14:33:16.853201 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.853204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.853503 (+ 299us) server_negotiation.cc:407] Connection header received
0811 14:33:16.854107 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.854108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.854122 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.854180 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:16.855385 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.856198 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.857593 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.857833 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.857937 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.861727 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:16.861733 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:16.861733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:16.861754 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:16.863972 (+ 2218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.864918 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.864922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.864922 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.865086 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.865673 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.865675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.865676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.865861 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:16.865940 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.866341 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.866463 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:33:16.866739 8622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.853134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:41293)
0811 14:33:16.853331 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:16.853381 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:16.853962 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:16.854277 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:16.854284 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:16.854975 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:16.855248 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.855254 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.856360 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.856362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.857472 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.857477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.857962 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.857964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.858129 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.859033 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:16.859051 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:16.861515 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:16.864169 (+ 2654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:16.864199 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:16.864204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:16.864605 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:16.865200 (+ 595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:16.865202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:16.865203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:16.865572 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:16.865943 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:16.865952 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:16.866139 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 14:33:16.866252 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 14:33:16.866474 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260811 14:33:16.867805 8620 heartbeater.cc:347] Connected to a master server at 127.23.186.254:35117
I20260811 14:33:16.867940 8620 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:16.868151 8620 heartbeater.cc:511] Master 127.23.186.254:35117 requested a full tablet report, sending...
I20260811 14:33:16.868620 8078 ts_manager.cc:194] Registered new tserver with Master: a88030d578454489985b6252c4dab642 (127.23.186.195:34849)
I20260811 14:33:16.869096 8078 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:41293
I20260811 14:33:16.877516 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:33:16.888513 8129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.879405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.23.186.254:35117)
0811 14:33:16.879611 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.879615 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.879763 (+ 148us) server_negotiation.cc:407] Connection header received
0811 14:33:16.879876 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.879877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.879892 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.879981 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:16.880690 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.881586 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.882785 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.883061 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.883165 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.885131 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:16.885135 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:16.885135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:16.885147 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:16.887037 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.887512 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.887513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.887514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.887583 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:16.887804 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:16.887805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:16.887805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:16.887999 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:16.888123 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.888280 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.888394 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260811 14:33:16.892035 8078 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:33:16.895191 8078 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:33:16.905042 8631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.899872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46117 (local address 127.0.0.1:48412)
0811 14:33:16.900135 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:16.900190 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:16.900279 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:16.900811 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:16.900814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:16.900818 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:16.900971 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.900973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.902355 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.902356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.903600 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.903603 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.904642 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.904643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.904769 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.904776 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:33:16.904846 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:33:16.904936 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":142,"thread_start_us":83,"threads_started":1}
I20260811 14:33:16.905055 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.899987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60090 (local address 127.23.186.194:33789)
0811 14:33:16.900435 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.900441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.900452 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:33:16.900602 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.900605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.900624 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.900732 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:16.901052 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.902260 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.903684 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.904480 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.904585 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.904657 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.904888 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.904951 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":126,"threads_started":1}
I20260811 14:33:16.905149 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.900055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.23.186.193:46117)
0811 14:33:16.900546 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.900551 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.900561 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:33:16.900633 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.900635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.900654 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.900921 (+ 267us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:16.901043 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.902222 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.903706 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.904524 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.904911 (+ 387us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.904981 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.905008 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.905057 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":95,"threads_started":1}
I20260811 14:33:16.905164 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.900361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43766 (local address 127.23.186.195:34849)
0811 14:33:16.900580 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:16.900593 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:16.900604 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:33:16.900647 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:16.900649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:16.900667 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:16.900775 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:16.901017 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.901843 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.903706 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.904524 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.904610 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.904672 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:16.904992 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 14:33:16.905048 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":101,"threads_started":1}
I20260811 14:33:16.905683 8635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.900231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:34849 (local address 127.0.0.1:43766)
0811 14:33:16.900381 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:16.900392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:16.900449 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:16.900852 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:16.900853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:16.900855 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:16.900921 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.900924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.902001 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.902002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.903578 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.903583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.904731 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.904732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.904811 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.904816 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:33:16.904869 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 14:33:16.904931 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":51,"thread_start_us":82,"threads_started":1}
I20260811 14:33:16.905895 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:16.899898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33789 (local address 127.0.0.1:60090)
0811 14:33:16.900205 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:16.900218 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:16.900279 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:16.900832 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:16.900832 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:16.900834 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:16.900956 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.900959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.902383 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.902384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.903560 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:16.903566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:16.904585 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:16.904587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:16.904730 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:16.904751 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 14:33:16.904843 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 14:33:16.905839 (+ 996us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":78,"thread_start_us":185,"threads_started":1}
I20260811 14:33:16.906284 8229 tablet_service.cc:1467] Processing CreateTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 (DEFAULT_TABLE table=test-table [id=641d48ad11a74c16aa895f10b81d6464]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:16.906710 8229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349d9cb8f1b748ca8f5d8203d57e6065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:16.909965 8545 tablet_service.cc:1467] Processing CreateTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 (DEFAULT_TABLE table=test-table [id=641d48ad11a74c16aa895f10b81d6464]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:16.909965 8385 tablet_service.cc:1467] Processing CreateTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 (DEFAULT_TABLE table=test-table [id=641d48ad11a74c16aa895f10b81d6464]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:16.910454 8545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349d9cb8f1b748ca8f5d8203d57e6065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:16.910454 8385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 349d9cb8f1b748ca8f5d8203d57e6065. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:16.912073 8637 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Bootstrap starting.
I20260811 14:33:16.913287 8637 tablet_bootstrap.cc:654] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:16.914182 8637 log.cc:824] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:16.914827 8640 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Bootstrap starting.
I20260811 14:33:16.914827 8639 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Bootstrap starting.
I20260811 14:33:16.915473 8637 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: No bootstrap required, opened a new log
I20260811 14:33:16.915596 8637 ts_tablet_manager.cc:1397] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 14:33:16.915637 8640 tablet_bootstrap.cc:654] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:16.915800 8639 tablet_bootstrap.cc:654] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:16.916474 8639 log.cc:824] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:16.916474 8640 log.cc:824] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:16.917490 8639 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: No bootstrap required, opened a new log
I20260811 14:33:16.917500 8640 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: No bootstrap required, opened a new log
I20260811 14:33:16.917582 8639 ts_tablet_manager.cc:1397] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 14:33:16.917585 8640 ts_tablet_manager.cc:1397] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 14:33:16.918030 8637 raft_consensus.cc:348] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:16.918231 8637 raft_consensus.cc:374] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:16.918295 8637 raft_consensus.cc:729] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7473880090e544dda868033a4ce5370a, State: Initialized, Role: FOLLOWER
I20260811 14:33:16.918450 8637 consensus_queue.cc:260] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:16.918793 8637 ts_tablet_manager.cc:1428] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:33:16.918969 8308 heartbeater.cc:503] Master 127.23.186.254:35117 was elected leader, sending a full tablet report...
I20260811 14:33:16.919793 8639 raft_consensus.cc:348] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:16.919793 8640 raft_consensus.cc:348] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:16.919997 8640 raft_consensus.cc:374] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:16.919996 8639 raft_consensus.cc:374] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:16.920058 8639 raft_consensus.cc:729] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0543a258afc4b59b7a2be4b0d950d94, State: Initialized, Role: FOLLOWER
I20260811 14:33:16.920081 8640 raft_consensus.cc:729] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88030d578454489985b6252c4dab642, State: Initialized, Role: FOLLOWER
I20260811 14:33:16.920517 8640 consensus_queue.cc:260] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:16.920517 8639 consensus_queue.cc:260] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:16.921252 8640 ts_tablet_manager.cc:1428] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20260811 14:33:16.921252 8639 ts_tablet_manager.cc:1428] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 14:33:16.921268 8466 heartbeater.cc:503] Master 127.23.186.254:35117 was elected leader, sending a full tablet report...
I20260811 14:33:16.921290 8620 heartbeater.cc:503] Master 127.23.186.254:35117 was elected leader, sending a full tablet report...
W20260811 14:33:16.947628 8309 tablet.cc:2369] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:17.102877 8621 tablet.cc:2369] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:17.139210 8644 raft_consensus.cc:482] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:17.139416 8644 raft_consensus.cc:504] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:17.140300 8644 leader_election.cc:288] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7473880090e544dda868033a4ce5370a (127.23.186.193:46117), a88030d578454489985b6252c4dab642 (127.23.186.195:34849)
I20260811 14:33:17.144793 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.140665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:37435 (local address 127.23.186.195:34849)
0811 14:33:17.140750 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:17.140752 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:17.141012 (+ 260us) server_negotiation.cc:407] Connection header received
0811 14:33:17.141082 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:17.141083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:17.141097 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:17.141161 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:17.141372 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.141931 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.143636 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.144296 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.144396 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.144412 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:17.144592 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 14:33:17.144649 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:33:17.144809 8646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.140508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:34849 (local address 127.23.186.194:37435)
0811 14:33:17.140938 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:17.140952 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:17.141015 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:17.141186 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:17.141187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:17.141189 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:17.141282 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.141284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.142120 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.142121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.143468 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.143477 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.144396 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.144398 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.144494 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.144500 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:33:17.144580 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:33:17.144649 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":187,"thread_start_us":137,"threads_started":1}
I20260811 14:33:17.144970 8634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.140637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:47133 (local address 127.23.186.193:46117)
0811 14:33:17.140755 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:17.140759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:17.140795 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:33:17.140838 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:17.140839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:17.140859 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:17.140924 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:17.141162 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.141940 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.143752 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.144273 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.144465 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.144480 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:17.144803 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 14:33:17.144866 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 14:33:17.144975 8468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.140508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46117 (local address 127.23.186.194:47133)
0811 14:33:17.140617 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:17.140638 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:17.140730 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:17.140954 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:17.140956 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:17.140958 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:17.141068 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.141071 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.142068 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.142069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.143476 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.143477 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.144473 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.144477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.144628 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.144635 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:33:17.144779 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 14:33:17.144880 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":40}
I20260811 14:33:17.145750 8569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349d9cb8f1b748ca8f5d8203d57e6065" candidate_uuid: "c0543a258afc4b59b7a2be4b0d950d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88030d578454489985b6252c4dab642" is_pre_election: true
I20260811 14:33:17.145766 8251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349d9cb8f1b748ca8f5d8203d57e6065" candidate_uuid: "c0543a258afc4b59b7a2be4b0d950d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7473880090e544dda868033a4ce5370a" is_pre_election: true
I20260811 14:33:17.146016 8569 raft_consensus.cc:2454] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0543a258afc4b59b7a2be4b0d950d94 in term 0.
I20260811 14:33:17.146010 8251 raft_consensus.cc:2454] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c0543a258afc4b59b7a2be4b0d950d94 in term 0.
I20260811 14:33:17.146351 8334 leader_election.cc:302] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94; no voters:
I20260811 14:33:17.146570 8644 raft_consensus.cc:2781] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:33:17.146652 8644 raft_consensus.cc:482] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:33:17.146692 8644 raft_consensus.cc:3037] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:17.147938 8644 raft_consensus.cc:504] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:17.148218 8644 leader_election.cc:288] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [CANDIDATE]: Term 1 election: Requested vote from peers 7473880090e544dda868033a4ce5370a (127.23.186.193:46117), a88030d578454489985b6252c4dab642 (127.23.186.195:34849)
I20260811 14:33:17.148559 8251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349d9cb8f1b748ca8f5d8203d57e6065" candidate_uuid: "c0543a258afc4b59b7a2be4b0d950d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7473880090e544dda868033a4ce5370a"
I20260811 14:33:17.148567 8569 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "349d9cb8f1b748ca8f5d8203d57e6065" candidate_uuid: "c0543a258afc4b59b7a2be4b0d950d94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a88030d578454489985b6252c4dab642"
I20260811 14:33:17.148687 8251 raft_consensus.cc:3037] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:17.148685 8569 raft_consensus.cc:3037] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:17.149752 8251 raft_consensus.cc:2454] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0543a258afc4b59b7a2be4b0d950d94 in term 1.
I20260811 14:33:17.149863 8569 raft_consensus.cc:2454] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c0543a258afc4b59b7a2be4b0d950d94 in term 1.
W20260811 14:33:17.149886 8467 tablet.cc:2369] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:17.150120 8334 leader_election.cc:302] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94; no voters:
I20260811 14:33:17.150254 8644 raft_consensus.cc:2781] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:17.150460 8644 raft_consensus.cc:686] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 LEADER]: Becoming Leader. State: Replica: c0543a258afc4b59b7a2be4b0d950d94, State: Running, Role: LEADER
I20260811 14:33:17.150584 8644 consensus_queue.cc:237] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:17.151563 8076 catalog_manager.cc:5495] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 reported cstate change: term changed from 0 to 1, leader changed from <none> to c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "c0543a258afc4b59b7a2be4b0d950d94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } health_report { overall_health: UNKNOWN } } }
I20260811 14:33:17.205032 8633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.200397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.23.186.194:33789)
0811 14:33:17.200520 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:17.200523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:17.200532 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:17.200595 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:17.200596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:17.200630 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:17.200920 (+ 290us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:17.201197 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.201820 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.203544 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.203789 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.203869 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.204353 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:17.204510 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:33:17.204604 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:17.204834 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 14:33:17.204869 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 14:33:17.207439 8251 raft_consensus.cc:1261] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Refusing update from remote peer c0543a258afc4b59b7a2be4b0d950d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:17.207441 8569 raft_consensus.cc:1261] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Refusing update from remote peer c0543a258afc4b59b7a2be4b0d950d94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:17.207955 8644 consensus_queue.cc:1035] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:17.208128 8647 consensus_queue.cc:1035] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:17.211547 8653 mvcc.cc:205] Tried to move back new op lower bound from 7317335233358053376 to 7317335233129598976. Current Snapshot: MvccSnapshot[applied={T|T < 7317335233358053376}]
I20260811 14:33:17.212296 8657 mvcc.cc:205] Tried to move back new op lower bound from 7317335233358053376 to 7317335233129598976. Current Snapshot: MvccSnapshot[applied={T|T < 7317335233358053376 or (T in {7317335233358053376})}]
I20260811 14:33:17.215549 8076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:17.215842 8076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:17.217404 8076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Sending DeleteTablet for 3 replicas of tablet 349d9cb8f1b748ca8f5d8203d57e6065
I20260811 14:33:17.218180 8385 tablet_service.cc:1514] Processing DeleteTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:17 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:60090
I20260811 14:33:17.218233 8545 tablet_service.cc:1514] Processing DeleteTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:17 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43766
I20260811 14:33:17.218386 8229 tablet_service.cc:1514] Processing DeleteTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:17 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:48412
I20260811 14:33:17.218771 8667 tablet_replica.cc:330] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: stopping tablet replica
I20260811 14:33:17.218976 8665 tablet_replica.cc:330] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: stopping tablet replica
I20260811 14:33:17.218972 8667 raft_consensus.cc:2229] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:17.218991 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8157
I20260811 14:33:17.219172 8666 tablet_replica.cc:330] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: stopping tablet replica
I20260811 14:33:17.219192 8667 raft_consensus.cc:2258] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:17.219506 8665 raft_consensus.cc:2229] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:17.219667 8666 raft_consensus.cc:2229] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:33:17.220000 8666 raft_consensus.cc:2258] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:17.220208 8665 raft_consensus.cc:2258] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:17.223300 8665 ts_tablet_manager.cc:1905] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:17.224308 8666 ts_tablet_manager.cc:1905] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:17.227272 8666 ts_tablet_manager.cc:1918] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:17.229913 8665 ts_tablet_manager.cc:1918] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:17.230106 8665 log.cc:1196] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/349d9cb8f1b748ca8f5d8203d57e6065
I20260811 14:33:17.230453 8665 ts_tablet_manager.cc:1939] T 349d9cb8f1b748ca8f5d8203d57e6065 P a88030d578454489985b6252c4dab642: Deleting consensus metadata
I20260811 14:33:17.230576 8666 log.cc:1196] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/349d9cb8f1b748ca8f5d8203d57e6065
I20260811 14:33:17.231717 8666 ts_tablet_manager.cc:1939] T 349d9cb8f1b748ca8f5d8203d57e6065 P c0543a258afc4b59b7a2be4b0d950d94: Deleting consensus metadata
I20260811 14:33:17.232456 8061 catalog_manager.cc:4841] TS c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): tablet 349d9cb8f1b748ca8f5d8203d57e6065 (table test-table [id=641d48ad11a74c16aa895f10b81d6464]) successfully deleted
I20260811 14:33:17.234736 8061 catalog_manager.cc:4841] TS a88030d578454489985b6252c4dab642 (127.23.186.195:34849): tablet 349d9cb8f1b748ca8f5d8203d57e6065 (table test-table [id=641d48ad11a74c16aa895f10b81d6464]) successfully deleted
W20260811 14:33:17.243050 8063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:46117 (error 108)
I20260811 14:33:17.243297 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8313
I20260811 14:33:17.246407 8632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.246219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46117 (local address 127.0.0.1:48428)
0811 14:33:17.246298 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:17.246352 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:46117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 14:33:17.246594 8063 catalog_manager.cc:4584] TS 7473880090e544dda868033a4ce5370a (127.23.186.193:46117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 349d9cb8f1b748ca8f5d8203d57e6065: Network error: Client connection negotiation failed: client connection to 127.23.186.193:46117: connect: Connection refused (error 111)
I20260811 14:33:17.252859 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8471
I20260811 14:33:17.265784 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8034
I20260811 14:33:17.276976 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35117
--webserver_interface=127.23.186.254
--webserver_port=34437
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:35117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:17.394227 8669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:17.394634 8669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:17.394685 8669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:17.396396 8669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:17.396494 8669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:17.396523 8669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:17.396545 8669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:17.396581 8669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:17.399021 8669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:35117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=34437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8669
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:17.399490 8669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:17.399852 8669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:17.415903 8675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.416082 8676 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.416103 8678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:17.416234 8669 server_base.cc:1034] running on GCE node
I20260811 14:33:17.416816 8669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:17.417277 8669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:17.418550 8669 hybrid_clock.cc:648] HybridClock initialized: now 1786458797418497 us; error 72 us; skew 500 ppm
Aug 11 14:33:17 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:17.422989 8669 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:17.423974 8669 webserver.cc:460] Webserver started at http://127.23.186.254:34437/ using document root <none> and password file <none>
I20260811 14:33:17.424525 8669 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:17.424659 8669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:17.427127 8669 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:33:17.428447 8685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:17.428753 8669 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:17.428879 8669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "cea0c0279558468aae45096a18cfe1a4"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "359382dcf95c40928dfb282da4fdea03"
server_key_iv: "13aae4bd9cae2637d7a74124cf6b07ea"
server_key_version: "encryptionkey@0"
I20260811 14:33:17.429415 8669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:17.465315 8669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:17.468194 8669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:17.468344 8669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:17.474754 8669 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:35117
I20260811 14:33:17.474871 8744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:35117 every 8 connection(s)
I20260811 14:33:17.475442 8669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:17.477563 8745 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:33:17.478132 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8669
I20260811 14:33:17.478531 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:46117
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:35117
--webserver_port=34827
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:17.478556 8745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap starting.
I20260811 14:33:17.488652 8745 log.cc:824] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:17.491123 8745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:17.491535 8745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap complete.
I20260811 14:33:17.494360 8745 raft_consensus.cc:348] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:17.494709 8745 raft_consensus.cc:729] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0c0279558468aae45096a18cfe1a4, State: Initialized, Role: FOLLOWER
I20260811 14:33:17.494937 8745 consensus_queue.cc:260] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:17.495030 8745 raft_consensus.cc:388] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:17.495105 8745 raft_consensus.cc:482] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:17.495165 8745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:33:17.496505 8745 raft_consensus.cc:504] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:17.496672 8745 leader_election.cc:302] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cea0c0279558468aae45096a18cfe1a4; no voters:
I20260811 14:33:17.496958 8745 leader_election.cc:288] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:33:17.497006 8750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:33:17.497151 8750 raft_consensus.cc:686] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 LEADER]: Becoming Leader. State: Replica: cea0c0279558468aae45096a18cfe1a4, State: Running, Role: LEADER
I20260811 14:33:17.497269 8750 consensus_queue.cc:237] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:17.497324 8745 sys_catalog.cc:564] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:17.497581 8751 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cea0c0279558468aae45096a18cfe1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } } }
I20260811 14:33:17.497715 8751 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:17.498085 8752 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cea0c0279558468aae45096a18cfe1a4. Latest consensus state: current_term: 2 leader_uuid: "cea0c0279558468aae45096a18cfe1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } } }
I20260811 14:33:17.498240 8752 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:17.498308 8754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:17.499333 8754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:17.499468 8754 catalog_manager.cc:1259] Loaded cluster ID: 07157ac69f5d4e2c8353e9dcd0dfe016
I20260811 14:33:17.499512 8754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:17.499894 8754 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 14:33:17.504587 8754 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 14:33:17.507776 8754 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:17.508117 8754 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Loaded TSK: 0
I20260811 14:33:17.508394 8754 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:33:17.587782 8748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:17.588099 8748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:17.588150 8748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:17.589733 8748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:17.589829 8748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:17.589883 8748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:17.591742 8748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:46117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=34827
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:17.592135 8748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:17.592447 8748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:17.601393 8779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.601415 8777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.601541 8781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:17.601976 8748 server_base.cc:1034] running on GCE node
I20260811 14:33:17.602306 8748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:17.602712 8748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:17.603935 8748 hybrid_clock.cc:648] HybridClock initialized: now 1786458797603900 us; error 44 us; skew 500 ppm
Aug 11 14:33:17 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:17.607553 8748 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:17.608256 8748 webserver.cc:460] Webserver started at http://127.23.186.193:34827/ using document root <none> and password file <none>
I20260811 14:33:17.608537 8748 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:17.608634 8748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:17.610402 8748 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:17.611212 8789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:17.611392 8748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:17.611515 8748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "7473880090e544dda868033a4ce5370a"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "c0c9bdc4c166eefa726dc4a1fc3c3568"
server_key_iv: "ef61061aa4fea791888b7cd4e51f8f38"
server_key_version: "encryptionkey@0"
I20260811 14:33:17.612023 8748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:17.633598 8748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:17.636921 8748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:17.637105 8748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:17.637480 8748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:17.638144 8798 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:33:17.639847 8748 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:33:17.639925 8748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:33:17.639953 8748 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:33:17.640892 8748 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:33:17.640947 8748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:17.641006 8798 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Bootstrap starting.
I20260811 14:33:17.651358 8748 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:46117
I20260811 14:33:17.651983 8748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:17.653007 8921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:46117 every 8 connection(s)
I20260811 14:33:17.655545 8798 log.cc:824] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:17.656939 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8748
I20260811 14:33:17.657305 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:33789
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:35117
--webserver_port=40091
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:17.658941 8798 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:17.659299 8798 tablet_bootstrap.cc:492] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Bootstrap complete.
I20260811 14:33:17.660632 8798 ts_tablet_manager.cc:1397] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.008s
I20260811 14:33:17.662326 8798 raft_consensus.cc:348] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:17.662770 8798 raft_consensus.cc:729] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7473880090e544dda868033a4ce5370a, State: Initialized, Role: FOLLOWER
I20260811 14:33:17.662951 8798 consensus_queue.cc:260] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } } peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } }
I20260811 14:33:17.663316 8798 ts_tablet_manager.cc:1428] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
Aug 11 14:33:17 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:17.669780 8924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.652969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:57993)
0811 14:33:17.655933 (+ 2964us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:17.655993 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:17.656625 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:17.657615 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:17.657620 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:17.658060 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:17.658397 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.658405 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.660079 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.660080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.661094 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.661097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.661802 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.661804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.661950 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.662673 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:17.662685 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:17.664516 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:17.667368 (+ 2852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:17.667384 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:17.667388 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:17.667820 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:17.668187 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:17.668188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:17.668188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:17.668350 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:17.669043 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:17.669058 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:17.669228 (+ 170us) client_negotiation.cc:769] Sending connection context
0811 14:33:17.669366 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 14:33:17.669596 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2901,"thread_start_us":116,"threads_started":1}
I20260811 14:33:17.669950 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.653069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:57993 (local address 127.23.186.254:35117)
0811 14:33:17.656945 (+ 3876us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:17.656957 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:17.657002 (+ 45us) server_negotiation.cc:407] Connection header received
0811 14:33:17.657204 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:17.657212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:17.657303 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:17.657454 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:17.658594 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.659945 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.661196 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.661626 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.661809 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.664682 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:17.664687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:17.664729 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:17.664747 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:17.667215 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:17.667940 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:17.667944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:17.667945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:17.668095 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:17.668451 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:17.668452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:17.668453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:17.668902 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:17.669006 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:17.669489 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 14:33:17.669688 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3808,"thread_start_us":102,"threads_started":1}
I20260811 14:33:17.670665 8922 heartbeater.cc:347] Connected to a master server at 127.23.186.254:35117
I20260811 14:33:17.670797 8922 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:17.671036 8922 heartbeater.cc:511] Master 127.23.186.254:35117 requested a full tablet report, sending...
I20260811 14:33:17.672081 8705 ts_manager.cc:194] Registered new tserver with Master: 7473880090e544dda868033a4ce5370a (127.23.186.193:46117)
I20260811 14:33:17.673321 8705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:57993
I20260811 14:33:17.674494 8922 heartbeater.cc:503] Master 127.23.186.254:35117 was elected leader, sending a full tablet report...
I20260811 14:33:17.682188 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.672836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46117 (local address 127.0.0.1:48438)
0811 14:33:17.673059 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:17.673105 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:17.673184 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:17.677342 (+ 4158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:17.677349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:17.677355 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:17.677476 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.677480 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.678836 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.678838 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.680420 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.680427 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.681682 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.681683 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.681818 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.681846 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 14:33:17.681969 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 14:33:17.682072 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260811 14:33:17.682309 8930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.672900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.23.186.193:46117)
0811 14:33:17.677001 (+ 4101us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:17.677010 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:17.677029 (+ 19us) server_negotiation.cc:407] Connection header received
0811 14:33:17.677079 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:17.677081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:17.677106 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:17.677293 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:17.677689 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.678684 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.680638 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.681523 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.681648 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.681728 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:17.682098 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 14:33:17.682190 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4030,"thread_start_us":125,"threads_started":1}
I20260811 14:33:17.683035 8846 tablet_service.cc:1514] Processing DeleteTablet for tablet 349d9cb8f1b748ca8f5d8203d57e6065 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:17 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:48438
I20260811 14:33:17.683797 8931 tablet_replica.cc:330] stopping tablet replica
I20260811 14:33:17.683951 8931 raft_consensus.cc:2229] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:17.684048 8931 raft_consensus.cc:2258] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:17.684439 8931 ts_tablet_manager.cc:1905] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:17.687673 8931 ts_tablet_manager.cc:1918] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:17.687808 8931 log.cc:1196] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/349d9cb8f1b748ca8f5d8203d57e6065
I20260811 14:33:17.688066 8931 ts_tablet_manager.cc:1939] T 349d9cb8f1b748ca8f5d8203d57e6065 P 7473880090e544dda868033a4ce5370a: Deleting consensus metadata
I20260811 14:33:17.689004 8691 catalog_manager.cc:4841] TS 7473880090e544dda868033a4ce5370a (127.23.186.193:46117): tablet 349d9cb8f1b748ca8f5d8203d57e6065 (table test-table [id=641d48ad11a74c16aa895f10b81d6464]) successfully deleted
W20260811 14:33:17.768258 8927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:17.768623 8927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:17.768671 8927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:17.770587 8927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:17.770689 8927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:17.770731 8927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:17.773351 8927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:33789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=40091
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:17.773880 8927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:17.774250 8927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:17.781716 8938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.781705 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.781705 8937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:17.782003 8927 server_base.cc:1034] running on GCE node
I20260811 14:33:17.782393 8927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:17.782750 8927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:17.783995 8927 hybrid_clock.cc:648] HybridClock initialized: now 1786458797783966 us; error 57 us; skew 500 ppm
Aug 11 14:33:17 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:17.787128 8927 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:17.787779 8927 webserver.cc:460] Webserver started at http://127.23.186.194:40091/ using document root <none> and password file <none>
I20260811 14:33:17.788097 8927 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:17.788206 8927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:17.790009 8927 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:17.790839 8949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:17.791100 8927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:17.791229 8927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "c0543a258afc4b59b7a2be4b0d950d94"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "2fc8f087f4e16e514266c9380a10a664"
server_key_iv: "9585807f375cb67009a6e08af3a8590e"
server_key_version: "encryptionkey@0"
I20260811 14:33:17.791676 8927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:17.817622 8927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:17.821471 8927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:17.821714 8927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:17.822011 8927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:17.822454 8927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:17.822517 8927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:17.822572 8927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:17.822620 8927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:17.829793 8927 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33789
I20260811 14:33:17.829878 9077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33789 every 8 connection(s)
I20260811 14:33:17.830346 8927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:33:17 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:17.838413 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 8927
I20260811 14:33:17.838876 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:34849
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:35117
--webserver_port=35551
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:17.842303 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.831335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:42817 (local address 127.23.186.254:35117)
0811 14:33:17.831400 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:17.831402 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:17.831646 (+ 244us) server_negotiation.cc:407] Connection header received
0811 14:33:17.832217 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:17.832219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:17.832235 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:17.832354 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:17.833701 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.834264 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.835406 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.835634 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.835706 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.838435 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:17.838439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:17.838440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:17.838461 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:17.840459 (+ 1998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:17.841104 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:17.841105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:17.841106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:17.841184 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:17.841487 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:17.841488 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:17.841488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:17.841643 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:17.841725 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:17.842075 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 14:33:17.842196 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:33:17.842535 9080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:17.831286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:42817)
0811 14:33:17.831501 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:17.831548 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:17.832092 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:17.832492 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:17.832497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:17.833305 (+ 808us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:17.833569 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.833577 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.834369 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.834370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.835292 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:17.835295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:17.835734 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:17.835735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:17.835851 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:17.836506 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:17.836515 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:17.838245 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:17.840620 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:17.840633 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:17.840637 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:17.841001 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:17.841260 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:17.841261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:17.841261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:17.841403 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:17.841726 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:17.841736 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:17.841915 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 14:33:17.842037 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 14:33:17.842249 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":135,"threads_started":1}
I20260811 14:33:17.843288 9078 heartbeater.cc:347] Connected to a master server at 127.23.186.254:35117
I20260811 14:33:17.843497 9078 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:17.843837 9078 heartbeater.cc:511] Master 127.23.186.254:35117 requested a full tablet report, sending...
I20260811 14:33:17.844496 8705 ts_manager.cc:194] Registered new tserver with Master: c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
I20260811 14:33:17.845031 8705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:42817
W20260811 14:33:17.943315 9082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:17.943660 9082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:17.943708 9082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:17.945305 9082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:17.945401 9082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:17.945462 9082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:17.947403 9082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:34849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=35551
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:17.947794 9082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:17.948082 9082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:17.955190 9088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.955348 9091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:17.955349 9087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:17.955861 9082 server_base.cc:1034] running on GCE node
I20260811 14:33:17.956188 9082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:17.956527 9082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:17.957779 9082 hybrid_clock.cc:648] HybridClock initialized: now 1786458797957750 us; error 49 us; skew 500 ppm
Aug 11 14:33:17 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:17.961846 9082 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:17.962502 9082 webserver.cc:460] Webserver started at http://127.23.186.195:35551/ using document root <none> and password file <none>
I20260811 14:33:17.962824 9082 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:17.962926 9082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:17.964663 9082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:33:17.965641 9098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:17.966089 9082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:17.966253 9082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "a88030d578454489985b6252c4dab642"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "67f84164bcd567be5bd713617b9c93c1"
server_key_iv: "5352614b9bd5a29aaa2cecbe0bb273a1"
server_key_version: "encryptionkey@0"
I20260811 14:33:17.966717 9082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:18.000509 9082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:18.003561 9082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:18.003715 9082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:18.004063 9082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:18.004441 9082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:18.004479 9082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:18.004504 9082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:18.004516 9082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:18.012137 9082 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:34849
I20260811 14:33:18.012207 9225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:34849 every 8 connection(s)
I20260811 14:33:18.012638 9082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:33:18.021147 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 9082
Aug 11 14:33:18 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458797, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:18.028919 8925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.014043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53153 (local address 127.23.186.254:35117)
0811 14:33:18.014151 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.014154 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.014476 (+ 322us) server_negotiation.cc:407] Connection header received
0811 14:33:18.015191 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.015193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.015209 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.015309 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:18.017171 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.017726 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.019315 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.019593 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.019706 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.023702 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:18.023706 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:18.023706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:18.023721 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:18.026257 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:18.027067 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:18.027082 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:18.027082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:18.027213 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:18.027688 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:18.027690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:18.027690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:18.027901 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:18.028012 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.028470 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.028804 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 14:33:18.029202 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.014028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:53153)
0811 14:33:18.014281 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.014341 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:18.015048 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:18.015614 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:18.015621 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:18.016508 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:18.016945 (+ 437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.016956 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.018011 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.018014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.019124 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.019130 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.019751 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.019752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.019942 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.021227 (+ 1285us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:18.021241 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:18.023445 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:18.026416 (+ 2971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:18.026443 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:18.026449 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:18.026919 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:18.027372 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:18.027373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:18.027373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:18.027578 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:18.028021 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:18.028044 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:18.028282 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 14:33:18.028429 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 14:33:18.028887 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":137,"threads_started":1}
I20260811 14:33:18.029838 9226 heartbeater.cc:347] Connected to a master server at 127.23.186.254:35117
I20260811 14:33:18.029999 9226 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:18.030289 9226 heartbeater.cc:511] Master 127.23.186.254:35117 requested a full tablet report, sending...
I20260811 14:33:18.030918 8705 ts_manager.cc:194] Registered new tserver with Master: a88030d578454489985b6252c4dab642 (127.23.186.195:34849)
I20260811 14:33:18.031363 8705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:53153
I20260811 14:33:18.033071 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.021763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.23.186.254:35117)
0811 14:33:18.021911 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.021913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.022017 (+ 104us) server_negotiation.cc:407] Connection header received
0811 14:33:18.022104 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.022105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.022118 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.022188 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:18.022936 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.023705 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.025448 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.025706 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.025787 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.028031 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:18.028036 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:18.028037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:18.028052 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:18.030892 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:18.031797 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:18.031799 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:18.031799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:18.031908 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:18.032289 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:18.032290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:18.032290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:18.032430 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:18.032756 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.032809 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.032928 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":84,"threads_started":1}
I20260811 14:33:18.034040 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:33:18.038038 9231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.034493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41238 (local address 127.23.186.254:35117)
0811 14:33:18.034535 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.034537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.034636 (+ 99us) server_negotiation.cc:407] Connection header received
0811 14:33:18.034720 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.034721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.034755 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.034815 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:18.034936 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.035443 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.036680 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.036996 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.037089 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.037475 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:18.037690 (+ 215us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:33:18.037798 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.037871 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.037919 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:33:18.038651 8705 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:41238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:33:18.040623 8705 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:33:18.044166 8846 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7d946ed1754ce99d2a476ffd93c195 (DEFAULT_TABLE table=test-table [id=40a136179dd0404f90ec0e1a42059571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:18.044452 8846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7d946ed1754ce99d2a476ffd93c195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:18.046535 8798 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Bootstrap starting.
I20260811 14:33:18.047210 8798 tablet_bootstrap.cc:654] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:18.048260 8798 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: No bootstrap required, opened a new log
I20260811 14:33:18.048364 8798 ts_tablet_manager.cc:1397] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:33:18.048522 8798 raft_consensus.cc:348] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.048623 8798 raft_consensus.cc:374] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:18.048749 8798 raft_consensus.cc:729] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7473880090e544dda868033a4ce5370a, State: Initialized, Role: FOLLOWER
I20260811 14:33:18.048825 8798 consensus_queue.cc:260] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.048961 8798 ts_tablet_manager.cc:1428] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:18.049217 8929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.044040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:34849 (local address 127.0.0.1:43776)
0811 14:33:18.044146 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.044162 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:18.044218 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:18.045120 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:18.045121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:18.045124 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:18.045189 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.045191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.046637 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.046639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.047488 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.047492 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.048889 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.048891 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.048976 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.048983 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:33:18.049021 (+ 38us) client_negotiation.cc:240] Negotiation successful
0811 14:33:18.049106 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 14:33:18.049710 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.044578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43776 (local address 127.23.186.195:34849)
0811 14:33:18.044894 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.044901 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.044916 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:33:18.044967 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.044969 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.044988 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.045266 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:18.045357 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.046496 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.047857 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.048764 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.049495 (+ 731us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.049569 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.049596 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.049640 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":168,"threads_started":1}
I20260811 14:33:18.050158 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.044871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60100 (local address 127.23.186.194:33789)
0811 14:33:18.045163 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.045170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.045292 (+ 122us) server_negotiation.cc:407] Connection header received
0811 14:33:18.045414 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.045417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.045442 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.045550 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:18.045723 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.046615 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.048128 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.049102 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.049209 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.049304 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.049961 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.050029 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":151,"threads_started":1}
I20260811 14:33:18.050352 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.044302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33789 (local address 127.0.0.1:60100)
0811 14:33:18.045181 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.045197 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:18.045314 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:18.045544 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:18.045545 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:18.045548 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:18.045644 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.045647 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.046759 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.046760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.048019 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.048022 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.049209 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.049210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.049530 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.049537 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 14:33:18.049603 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 14:33:18.049680 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":839,"thread_start_us":174,"threads_started":1}
I20260811 14:33:18.050593 9150 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7d946ed1754ce99d2a476ffd93c195 (DEFAULT_TABLE table=test-table [id=40a136179dd0404f90ec0e1a42059571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:18.051014 9150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7d946ed1754ce99d2a476ffd93c195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:18.051546 9002 tablet_service.cc:1467] Processing CreateTablet for tablet 3f7d946ed1754ce99d2a476ffd93c195 (DEFAULT_TABLE table=test-table [id=40a136179dd0404f90ec0e1a42059571]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:18.051870 9002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f7d946ed1754ce99d2a476ffd93c195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:18.054971 9241 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Bootstrap starting.
I20260811 14:33:18.055114 9242 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Bootstrap starting.
I20260811 14:33:18.055677 9241 tablet_bootstrap.cc:654] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:18.055805 9242 tablet_bootstrap.cc:654] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:18.056314 9241 log.cc:824] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:18.056447 9242 log.cc:824] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:18.057843 9241 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: No bootstrap required, opened a new log
I20260811 14:33:18.057963 9242 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: No bootstrap required, opened a new log
I20260811 14:33:18.057960 9241 ts_tablet_manager.cc:1397] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:33:18.058063 9242 ts_tablet_manager.cc:1397] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 14:33:18.059983 9241 raft_consensus.cc:348] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.060019 9242 raft_consensus.cc:348] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.060233 9241 raft_consensus.cc:374] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:18.060267 9241 raft_consensus.cc:729] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a88030d578454489985b6252c4dab642, State: Initialized, Role: FOLLOWER
I20260811 14:33:18.060271 9242 raft_consensus.cc:374] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:18.060319 9242 raft_consensus.cc:729] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0543a258afc4b59b7a2be4b0d950d94, State: Initialized, Role: FOLLOWER
I20260811 14:33:18.060465 9241 consensus_queue.cc:260] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.060632 9242 consensus_queue.cc:260] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.061097 9241 ts_tablet_manager.cc:1428] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260811 14:33:18.061184 9242 ts_tablet_manager.cc:1428] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260811 14:33:18.061411 9226 heartbeater.cc:503] Master 127.23.186.254:35117 was elected leader, sending a full tablet report...
I20260811 14:33:18.061408 9078 heartbeater.cc:503] Master 127.23.186.254:35117 was elected leader, sending a full tablet report...
W20260811 14:33:18.081178 9079 tablet.cc:2369] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:18.153864 8923 tablet.cc:2369] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:18.263584 9227 tablet.cc:2369] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:18.401087 9246 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:18.401269 9246 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.401713 9246 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789), 7473880090e544dda868033a4ce5370a (127.23.186.193:46117)
I20260811 14:33:18.406102 9228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.401950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46117 (local address 127.23.186.195:44157)
0811 14:33:18.402019 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.402050 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:18.402116 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:18.402496 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:18.402500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:18.402506 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:18.402771 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.402777 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.403782 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.403785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.404901 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.404906 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.405791 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.405793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.405908 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.405914 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:33:18.405983 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 14:33:18.406028 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 14:33:18.406205 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.402106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:44157 (local address 127.23.186.193:46117)
0811 14:33:18.402250 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.402253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.402264 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:33:18.402304 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.402304 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.402318 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.402400 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:18.402943 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.403635 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.405029 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.405576 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.405665 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.405675 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.406029 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.406097 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":67,"threads_started":1}
I20260811 14:33:18.406332 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.401975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33789 (local address 127.23.186.195:49489)
0811 14:33:18.402247 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.402258 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:18.402342 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:18.402911 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:18.402912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:18.402915 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:18.403036 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.403038 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.404109 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.404110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.405098 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:18.405101 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.405999 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.406001 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:18.406093 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.406099 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 14:33:18.406175 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 14:33:18.406218 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":21,"thread_start_us":129,"threads_started":1}
I20260811 14:33:18.406363 9239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.402107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:49489 (local address 127.23.186.194:33789)
0811 14:33:18.402167 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.402170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.402481 (+ 311us) server_negotiation.cc:407] Connection header received
0811 14:33:18.402532 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.402534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.402559 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.402803 (+ 244us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:18.403265 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.403898 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.405257 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.405794 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.405888 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.405899 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.406208 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.406270 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 14:33:18.406595 8862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7d946ed1754ce99d2a476ffd93c195" candidate_uuid: "a88030d578454489985b6252c4dab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7473880090e544dda868033a4ce5370a" is_pre_election: true
I20260811 14:33:18.406812 8862 raft_consensus.cc:2454] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88030d578454489985b6252c4dab642 in term 0.
I20260811 14:33:18.406855 9024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7d946ed1754ce99d2a476ffd93c195" candidate_uuid: "a88030d578454489985b6252c4dab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0543a258afc4b59b7a2be4b0d950d94" is_pre_election: true
I20260811 14:33:18.407071 9024 raft_consensus.cc:2454] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a88030d578454489985b6252c4dab642 in term 0.
I20260811 14:33:18.407150 9103 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7473880090e544dda868033a4ce5370a, a88030d578454489985b6252c4dab642; no voters:
I20260811 14:33:18.407354 9246 raft_consensus.cc:2781] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:33:18.407424 9246 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:33:18.407455 9246 raft_consensus.cc:3037] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:18.408349 9246 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.408545 9246 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [CANDIDATE]: Term 1 election: Requested vote from peers c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789), 7473880090e544dda868033a4ce5370a (127.23.186.193:46117)
I20260811 14:33:18.408802 9024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7d946ed1754ce99d2a476ffd93c195" candidate_uuid: "a88030d578454489985b6252c4dab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0543a258afc4b59b7a2be4b0d950d94"
I20260811 14:33:18.408854 8862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f7d946ed1754ce99d2a476ffd93c195" candidate_uuid: "a88030d578454489985b6252c4dab642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7473880090e544dda868033a4ce5370a"
I20260811 14:33:18.408907 9024 raft_consensus.cc:3037] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:18.408998 8862 raft_consensus.cc:3037] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:18.409916 9024 raft_consensus.cc:2454] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88030d578454489985b6252c4dab642 in term 1.
I20260811 14:33:18.409960 8862 raft_consensus.cc:2454] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a88030d578454489985b6252c4dab642 in term 1.
I20260811 14:33:18.410163 9101 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94; no voters:
I20260811 14:33:18.410279 9246 raft_consensus.cc:2781] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:18.410456 9246 raft_consensus.cc:686] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 1 LEADER]: Becoming Leader. State: Replica: a88030d578454489985b6252c4dab642, State: Running, Role: LEADER
I20260811 14:33:18.410593 9246 consensus_queue.cc:237] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.411754 8705 catalog_manager.cc:5495] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 reported cstate change: term changed from 0 to 1, leader changed from <none> to a88030d578454489985b6252c4dab642 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "a88030d578454489985b6252c4dab642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20260811 14:33:18.505221 9238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.501478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.23.186.195:34849)
0811 14:33:18.501535 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:18.501537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:18.501548 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:33:18.501608 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:18.501609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:18.501632 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:18.501726 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:18.502226 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.502857 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.503941 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:18.504272 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:18.504372 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:18.504764 (+ 392us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:18.504936 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:33:18.505029 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:18.505108 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 14:33:18.505143 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:33:18.508023 9024 raft_consensus.cc:1261] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Refusing update from remote peer a88030d578454489985b6252c4dab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:18.508023 8862 raft_consensus.cc:1261] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Refusing update from remote peer a88030d578454489985b6252c4dab642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:18.508558 9250 consensus_queue.cc:1035] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:18.508901 9246 consensus_queue.cc:1035] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:18.512091 9254 mvcc.cc:205] Tried to move back new op lower bound from 7317335238684180480 to 7317335238290505728. Current Snapshot: MvccSnapshot[applied={T|T < 7317335238684180480}]
I20260811 14:33:18.512498 9253 mvcc.cc:205] Tried to move back new op lower bound from 7317335238684180480 to 7317335238290505728. Current Snapshot: MvccSnapshot[applied={T|T < 7317335238684180480 or (T in {7317335238684180480})}]
I20260811 14:33:18.513208 9255 mvcc.cc:205] Tried to move back new op lower bound from 7317335238684180480 to 7317335238290505728. Current Snapshot: MvccSnapshot[applied={T|T < 7317335238684180480}]
I20260811 14:33:18.515326 8704 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:41238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:18.515489 8704 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:41238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:18.517218 8704 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Sending DeleteTablet for 3 replicas of tablet 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:18.517758 8846 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7d946ed1754ce99d2a476ffd93c195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:18 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:48438
I20260811 14:33:18.517860 9002 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7d946ed1754ce99d2a476ffd93c195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:18 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:60100
I20260811 14:33:18.518086 9150 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f7d946ed1754ce99d2a476ffd93c195 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:18 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:43776
I20260811 14:33:18.518249 9264 tablet_replica.cc:330] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: stopping tablet replica
I20260811 14:33:18.518396 9264 raft_consensus.cc:2229] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:18.518496 9263 tablet_replica.cc:330] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: stopping tablet replica
I20260811 14:33:18.518565 9263 raft_consensus.cc:2229] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:18.518569 9264 raft_consensus.cc:2258] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:18.518625 9263 raft_consensus.cc:2258] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:18.518640 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8748
I20260811 14:33:18.518788 9265 tablet_replica.cc:330] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: stopping tablet replica
I20260811 14:33:18.518903 9265 raft_consensus.cc:2229] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:33:18.519028 9265 raft_consensus.cc:2258] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:18.519220 9264 ts_tablet_manager.cc:1905] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:18.521894 9265 ts_tablet_manager.cc:1905] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:18.521880 9264 ts_tablet_manager.cc:1918] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:18.522017 9264 log.cc:1196] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:18.522429 9264 ts_tablet_manager.cc:1939] T 3f7d946ed1754ce99d2a476ffd93c195 P c0543a258afc4b59b7a2be4b0d950d94: Deleting consensus metadata
I20260811 14:33:18.523162 8688 catalog_manager.cc:4841] TS c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): tablet 3f7d946ed1754ce99d2a476ffd93c195 (table test-table [id=40a136179dd0404f90ec0e1a42059571]) successfully deleted
I20260811 14:33:18.523931 9265 ts_tablet_manager.cc:1918] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:18.524039 9265 log.cc:1196] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:18.524384 9265 ts_tablet_manager.cc:1939] T 3f7d946ed1754ce99d2a476ffd93c195 P a88030d578454489985b6252c4dab642: Deleting consensus metadata
I20260811 14:33:18.525408 8688 catalog_manager.cc:4841] TS a88030d578454489985b6252c4dab642 (127.23.186.195:34849): tablet 3f7d946ed1754ce99d2a476ffd93c195 (table test-table [id=40a136179dd0404f90ec0e1a42059571]) successfully deleted
W20260811 14:33:18.534224 8691 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:46117 (error 108)
I20260811 14:33:18.534660 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8927
I20260811 14:33:18.540338 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:18.534875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:46117 (local address 127.0.0.1:48446)
0811 14:33:18.540135 (+ 5260us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.540262 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:46117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5210}
W20260811 14:33:18.544996 8691 catalog_manager.cc:4584] TS 7473880090e544dda868033a4ce5370a (127.23.186.193:46117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f7d946ed1754ce99d2a476ffd93c195: Network error: Client connection negotiation failed: client connection to 127.23.186.193:46117: connect: Connection refused (error 111)
I20260811 14:33:18.549229 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 9082
I20260811 14:33:18.561053 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 8669
I20260811 14:33:18.571640 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35117
--webserver_interface=127.23.186.254
--webserver_port=34437
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:35117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:18.676961 9268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:18.677313 9268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:18.677371 9268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:18.679103 9268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:18.679214 9268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:18.679241 9268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:18.679266 9268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:18.679301 9268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:18.681588 9268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:35117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:35117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=34437
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:18.682015 9268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:18.682334 9268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:18.688846 9273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:18.688846 9274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:18.688982 9276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:18.689456 9268 server_base.cc:1034] running on GCE node
I20260811 14:33:18.689738 9268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:18.690083 9268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:18.691275 9268 hybrid_clock.cc:648] HybridClock initialized: now 1786458798691196 us; error 96 us; skew 500 ppm
Aug 11 14:33:18 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458798, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:18.695914 9268 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:18.696821 9268 webserver.cc:460] Webserver started at http://127.23.186.254:34437/ using document root <none> and password file <none>
I20260811 14:33:18.697186 9268 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:18.697278 9268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:18.699770 9268 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:33:18.701009 9285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:18.701455 9268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:18.701646 9268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "cea0c0279558468aae45096a18cfe1a4"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "359382dcf95c40928dfb282da4fdea03"
server_key_iv: "13aae4bd9cae2637d7a74124cf6b07ea"
server_key_version: "encryptionkey@0"
I20260811 14:33:18.702144 9268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:18.714272 9268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:18.718614 9268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:18.718899 9268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:18.725086 9268 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:35117
I20260811 14:33:18.725106 9341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:35117 every 8 connection(s)
I20260811 14:33:18.725924 9268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:18.728536 9342 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:33:18.729503 9342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap starting.
I20260811 14:33:18.731714 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 9268
I20260811 14:33:18.732172 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:46117
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:35117
--webserver_port=34827
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:18.743441 9342 log.cc:824] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:18.746670 9342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:18.747174 9342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4: Bootstrap complete.
I20260811 14:33:18.750698 9342 raft_consensus.cc:348] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:18.751118 9342 raft_consensus.cc:729] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: cea0c0279558468aae45096a18cfe1a4, State: Initialized, Role: FOLLOWER
I20260811 14:33:18.751402 9342 consensus_queue.cc:260] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:18.751531 9342 raft_consensus.cc:388] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:18.751590 9342 raft_consensus.cc:482] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:18.751677 9342 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:33:18.753506 9342 raft_consensus.cc:504] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:18.753705 9342 leader_election.cc:302] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cea0c0279558468aae45096a18cfe1a4; no voters:
I20260811 14:33:18.754036 9342 leader_election.cc:288] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 14:33:18.754096 9347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 14:33:18.754241 9347 raft_consensus.cc:686] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [term 3 LEADER]: Becoming Leader. State: Replica: cea0c0279558468aae45096a18cfe1a4, State: Running, Role: LEADER
I20260811 14:33:18.754375 9347 consensus_queue.cc:237] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } }
I20260811 14:33:18.754516 9342 sys_catalog.cc:564] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:18.754781 9347 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cea0c0279558468aae45096a18cfe1a4. Latest consensus state: current_term: 3 leader_uuid: "cea0c0279558468aae45096a18cfe1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } } }
I20260811 14:33:18.754889 9347 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:18.755036 9350 sys_catalog.cc:455] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "cea0c0279558468aae45096a18cfe1a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cea0c0279558468aae45096a18cfe1a4" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 35117 } } }
I20260811 14:33:18.755169 9353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:18.755249 9350 sys_catalog.cc:458] T 00000000000000000000000000000000 P cea0c0279558468aae45096a18cfe1a4 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:18.756580 9353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:18.756916 9353 catalog_manager.cc:1259] Loaded cluster ID: 07157ac69f5d4e2c8353e9dcd0dfe016
I20260811 14:33:18.757097 9353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 14:33:18.757488 9353 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9c7359d9ad google::LogMessage::Fail() at ??:0
@ 0x7f9c735a088a google::LogMessage::SendToLog() at ??:0
@ 0x7f9c7359d4de google::LogMessage::Flush() at ??:0
@ 0x7f9c7359ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9c7b4be0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f9c7b4c78b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9c7546b0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9c75460954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9c762c46db start_thread at ??:0
@ 0x7f9c7209b61f clone at ??:0
W20260811 14:33:18.853837 9345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:18.854205 9345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:18.854259 9345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:18.855818 9345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:18.855938 9345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:18.856005 9345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:18.858201 9345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:46117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=34827
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:18.858623 9345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:18.858963 9345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:33:18.866529 9345 server_base.cc:1034] running on GCE node
W20260811 14:33:18.866459 9391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:18.866575 9389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:18.866432 9387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:18.867309 9345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:18.867787 9345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:18.869051 9345 hybrid_clock.cc:648] HybridClock initialized: now 1786458798868985 us; error 78 us; skew 500 ppm
Aug 11 14:33:18 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458798, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:18.872437 9345 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:18.873189 9345 webserver.cc:460] Webserver started at http://127.23.186.193:34827/ using document root <none> and password file <none>
I20260811 14:33:18.873503 9345 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:18.873586 9345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:18.875439 9345 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:33:18.876747 9399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:18.877082 9345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:18.877257 9345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "7473880090e544dda868033a4ce5370a"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "c0c9bdc4c166eefa726dc4a1fc3c3568"
server_key_iv: "ef61061aa4fea791888b7cd4e51f8f38"
server_key_version: "encryptionkey@0"
I20260811 14:33:18.877934 9345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:18.899722 9345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:18.903844 9345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:18.904084 9345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:18.904433 9345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:18.905135 9406 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:33:18.906893 9345 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:33:18.906970 9345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:33:18.907001 9345 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:33:18.907861 9345 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:33:18.907922 9345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 14:33:18.907980 9406 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Bootstrap starting.
I20260811 14:33:18.921039 9345 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:46117
I20260811 14:33:18.921293 9531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:46117 every 8 connection(s)
I20260811 14:33:18.922058 9345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:18.922818 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 9345
I20260811 14:33:18.923266 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:33789
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:35117
--webserver_port=40091
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:18.927071 9406 log.cc:824] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:18.931308 9406 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:18.931754 9406 tablet_bootstrap.cc:492] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Bootstrap complete.
I20260811 14:33:18.933270 9406 ts_tablet_manager.cc:1397] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.014s
I20260811 14:33:18.935065 9406 raft_consensus.cc:348] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.935498 9406 raft_consensus.cc:729] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7473880090e544dda868033a4ce5370a, State: Initialized, Role: FOLLOWER
I20260811 14:33:18.935712 9406 consensus_queue.cc:260] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:18.936137 9406 ts_tablet_manager.cc:1428] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260811 14:33:19.028038 9536 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:33:18.926754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:46149)
0811 14:33:18.927442 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:18.927498 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:18.929876 (+ 2378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:19.027791 (+ 97915us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":133,"threads_started":1}
W20260811 14:33:19.028429 9533 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:35117 (0 consecutive failures): Network error: Failed to ping master at 127.23.186.254:35117: Client connection negotiation failed: client connection to 127.23.186.254:35117: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 14:33:19.029156 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.028983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:34697)
0811 14:33:19.029054 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.029120 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 14:33:19.029623 9536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.029510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:53931)
0811 14:33:19.029564 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.029599 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 14:33:19.029815 9533 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:33:19.038582 9535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:19.039021 9535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:19.039085 9535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:19.041220 9535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:19.041350 9535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:19.041420 9535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:19.043970 9535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:33789
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=40091
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:19.044458 9535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:19.044951 9535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:19.051966 9544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:19.051970 9543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:19.051966 9546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:19.052976 9535 server_base.cc:1034] running on GCE node
I20260811 14:33:19.053630 9535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:19.054121 9535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:19.055379 9535 hybrid_clock.cc:648] HybridClock initialized: now 1786458799055359 us; error 54 us; skew 500 ppm
Aug 11 14:33:19 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458799, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:19.059609 9535 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:19.060334 9535 webserver.cc:460] Webserver started at http://127.23.186.194:40091/ using document root <none> and password file <none>
I20260811 14:33:19.060655 9535 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:19.060803 9535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:19.063514 9535 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:33:19.064651 9555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:19.064966 9535 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:19.065116 9535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "c0543a258afc4b59b7a2be4b0d950d94"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "2fc8f087f4e16e514266c9380a10a664"
server_key_iv: "9585807f375cb67009a6e08af3a8590e"
server_key_version: "encryptionkey@0"
I20260811 14:33:19.065626 9535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:19.074888 9535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:19.078575 9535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:19.078893 9535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:19.079276 9535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:19.079777 9535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:19.079843 9535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:19.079919 9535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:19.079954 9535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:19.089701 9535 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:33789
I20260811 14:33:19.089758 9692 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:33789 every 8 connection(s)
I20260811 14:33:19.090339 9535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:33:19.092116 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.091485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:60473)
0811 14:33:19.091778 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.091905 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":124,"threads_started":1}
W20260811 14:33:19.092829 9694 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:35117 (0 consecutive failures): Network error: Failed to ping master at 127.23.186.254:35117: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
I20260811 14:33:19.093358 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.093186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:55485)
0811 14:33:19.093289 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.093331 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 14:33:19.093955 9696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.093736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:38669)
0811 14:33:19.093898 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.093930 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260811 14:33:19.094304 9694 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:33:19.094856 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 9535
I20260811 14:33:19.095170 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:34849
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:35117
--webserver_port=35551
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:34665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:19.203008 9697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:19.203584 9697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:19.203684 9697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:19.205545 9697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:19.205677 9697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:19.205741 9697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:19.209421 9697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:34665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:34849
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=35551
--tserver_master_addrs=127.23.186.254:35117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:19.209893 9697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:19.210235 9697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:19.217643 9705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:19.217613 9710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:19.217648 9706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:19.218185 9697 server_base.cc:1034] running on GCE node
I20260811 14:33:19.218523 9697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:19.218915 9697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:19.220135 9697 hybrid_clock.cc:648] HybridClock initialized: now 1786458799220062 us; error 111 us; skew 500 ppm
Aug 11 14:33:19 dist-test-slave-cbsf krb5kdc[8019](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458799, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:19.224543 9697 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:19.225240 9697 webserver.cc:460] Webserver started at http://127.23.186.195:35551/ using document root <none> and password file <none>
I20260811 14:33:19.225504 9697 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:19.225558 9697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:19.227420 9697 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:19.228554 9717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:19.228924 9697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:19.229046 9697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "a88030d578454489985b6252c4dab642"
format_stamp: "Formatted at 2026-08-11 14:33:16 on dist-test-slave-cbsf"
server_key: "67f84164bcd567be5bd713617b9c93c1"
server_key_iv: "5352614b9bd5a29aaa2cecbe0bb273a1"
server_key_version: "encryptionkey@0"
I20260811 14:33:19.229561 9697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:19.243914 9697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:19.246810 9697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:19.247033 9697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:19.247364 9697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:19.247741 9697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:19.247802 9697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:19.247850 9697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:19.247885 9697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:19.255029 9697 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:34849
I20260811 14:33:19.255231 9849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:34849 every 8 connection(s)
I20260811 14:33:19.255632 9697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:33:19.257609 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.256661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:33033)
0811 14:33:19.256957 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.257090 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":152,"threads_started":1}
W20260811 14:33:19.258039 9850 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:35117 (0 consecutive failures): Network error: Failed to ping master at 127.23.186.254:35117: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
I20260811 14:33:19.258627 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.258407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:51155)
0811 14:33:19.258513 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.258580 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 14:33:19.259258 9852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:19.259075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:47447)
0811 14:33:19.259183 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:19.259226 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 14:33:19.259480 9850 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:33:19.265091 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 9697
I20260811 14:33:20.030699 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.030359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:36429)
0811 14:33:20.030586 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:20.030642 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":77,"threads_started":1}
I20260811 14:33:20.095480 9856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.094955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:37487)
0811 14:33:20.095185 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:20.095359 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":97,"threads_started":1}
I20260811 14:33:20.260457 9857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.260055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:43209)
0811 14:33:20.260298 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:20.260356 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":139,"threads_started":1}
I20260811 14:33:20.443178 9858 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:20.443331 9858 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:20.443689 9858 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a88030d578454489985b6252c4dab642 (127.23.186.195:34849), c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
Aug 11 14:33:20 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458798, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.194@KRBTEST.COM
Aug 11 14:33:20 dist-test-slave-cbsf krb5kdc[8019](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458798, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.195@KRBTEST.COM
I20260811 14:33:20.464109 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.444180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:33789 (local address 127.23.186.193:53611)
0811 14:33:20.444440 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:20.444453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:20.444558 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:20.445375 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:20.445377 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:20.445860 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:20.446094 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:20.446102 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.447403 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.447405 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:20.448413 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:20.448416 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.449348 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.449350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:20.449507 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:20.455542 (+ 6035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:20.455563 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:20.458482 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:20.461744 (+ 3262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:20.461747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:20.461748 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:20.462216 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:20.462712 (+ 496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:20.462713 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:20.462714 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:20.462885 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:20.463473 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:20.463474 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:20.463586 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 14:33:20.463715 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 14:33:20.463986 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":47,"thread_start_us":97,"threads_started":1}
I20260811 14:33:20.464241 9861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.444265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:53611 (local address 127.23.186.194:33789)
0811 14:33:20.444413 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:20.444425 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:20.444543 (+ 118us) server_negotiation.cc:407] Connection header received
0811 14:33:20.445143 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:20.445146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:20.445190 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:20.445364 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:20.446257 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.447250 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.448629 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.449214 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.449352 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:20.458744 (+ 9392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:20.458753 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:20.458754 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:20.458783 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:20.461547 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:20.462412 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:20.462420 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:20.462421 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:20.462609 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:20.462970 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:20.462971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:20.462971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:20.463364 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:20.463456 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:20.463773 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 14:33:20.463964 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260811 14:33:20.464403 9855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.443999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:34849 (local address 127.23.186.193:48371)
0811 14:33:20.444073 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:20.444088 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:20.444163 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:20.445345 (+ 1182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:20.445349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:20.445860 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:20.446098 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:20.446104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.447482 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.447483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:20.448411 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:20.448416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.449360 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.449362 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:20.449503 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:20.455558 (+ 6055us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:20.455566 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:20.458623 (+ 3057us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:20.461474 (+ 2851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:20.461494 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:20.461500 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:20.462218 (+ 718us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:20.462713 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:20.462714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:20.462715 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:20.462895 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:20.463421 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:20.463432 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:20.463587 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 14:33:20.463716 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 14:33:20.464197 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":49}
I20260811 14:33:20.464672 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:20.444123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:48371 (local address 127.23.186.195:34849)
0811 14:33:20.444356 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:20.444367 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:20.444401 (+ 34us) server_negotiation.cc:407] Connection header received
0811 14:33:20.445043 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:20.445046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:20.445106 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:20.445250 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:20.446278 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.447293 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.448662 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:20.449233 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:20.449389 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:20.458834 (+ 9445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:20.458840 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:20.458840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:20.458855 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:20.461287 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:20.462430 (+ 1143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:20.462435 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:20.462435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:20.462602 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:20.463008 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:20.463009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:20.463009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:20.463333 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:20.463445 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:20.463933 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 14:33:20.464092 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
W20260811 14:33:20.465430 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
W20260811 14:33:20.465550 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a88030d578454489985b6252c4dab642 (127.23.186.195:34849): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:20.465605 9401 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7473880090e544dda868033a4ce5370a; no voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94
I20260811 14:33:20.465761 9858 raft_consensus.cc:2726] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:33:21.032004 9863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:21.031536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:36381)
0811 14:33:21.031829 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:21.031921 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260811 14:33:21.096990 9864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:21.096563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:48637)
0811 14:33:21.096849 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:21.096913 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":196,"threads_started":1}
I20260811 14:33:21.261658 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:21.261300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:46463)
0811 14:33:21.261530 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:21.261597 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":99,"threads_started":1}
I20260811 14:33:22.033480 9867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:22.032998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:43889)
0811 14:33:22.033324 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:22.033401 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":144,"threads_started":1}
I20260811 14:33:22.098457 9868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:22.098003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:47603)
0811 14:33:22.098309 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:22.098372 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":148,"threads_started":1}
I20260811 14:33:22.262735 9869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:22.262404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:42813)
0811 14:33:22.262623 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:22.262679 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":93,"threads_started":1}
I20260811 14:33:22.642963 9870 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:22.643105 9870 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:22.643550 9870 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a88030d578454489985b6252c4dab642 (127.23.186.195:34849), c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
W20260811 14:33:22.644341 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a88030d578454489985b6252c4dab642 (127.23.186.195:34849): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
W20260811 14:33:22.644471 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:22.644495 9401 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7473880090e544dda868033a4ce5370a; no voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94
I20260811 14:33:22.644675 9870 raft_consensus.cc:2726] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:33:23.035002 9871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:23.034579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:49549)
0811 14:33:23.034864 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:23.034924 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":156,"threads_started":1}
I20260811 14:33:23.099635 9872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:23.099310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:50625)
0811 14:33:23.099490 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:23.099555 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":96,"threads_started":1}
I20260811 14:33:23.263847 9873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:23.263543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:42555)
0811 14:33:23.263724 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:23.263776 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":82,"threads_started":1}
I20260811 14:33:24.036530 9874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:24.036050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:58361)
0811 14:33:24.036395 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:24.036454 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":165,"threads_started":1}
I20260811 14:33:24.101488 9875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:24.100643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:34499)
0811 14:33:24.101348 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:24.101427 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":328,"threads_started":1}
I20260811 14:33:24.257793 9876 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:24.257921 9876 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:24.258417 9876 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a88030d578454489985b6252c4dab642 (127.23.186.195:34849), c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
W20260811 14:33:24.259420 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
W20260811 14:33:24.259584 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a88030d578454489985b6252c4dab642 (127.23.186.195:34849): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:24.259608 9401 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7473880090e544dda868033a4ce5370a; no voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94
I20260811 14:33:24.259732 9876 raft_consensus.cc:2726] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:33:24.265518 9878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:24.265002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:44797)
0811 14:33:24.265343 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:24.265431 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":99,"threads_started":1}
I20260811 14:33:25.038120 9879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:25.037637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:34173)
0811 14:33:25.037954 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:25.038020 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":232,"threads_started":1}
I20260811 14:33:25.103016 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:25.102606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:57345)
0811 14:33:25.102882 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:25.102957 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":118,"threads_started":1}
I20260811 14:33:25.267050 9882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:25.266606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:44403)
0811 14:33:25.266905 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:25.266990 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":158,"threads_started":1}
I20260811 14:33:26.039975 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:26.039503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:40177)
0811 14:33:26.039817 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:26.039895 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":148,"threads_started":1}
I20260811 14:33:26.104303 9884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:26.103943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:49627)
0811 14:33:26.104194 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:26.104249 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":100,"threads_started":1}
I20260811 14:33:26.268488 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:26.267990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:56841)
0811 14:33:26.268331 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:26.268426 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":107,"threads_started":1}
I20260811 14:33:26.580178 9886 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:26.580353 9886 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:26.580834 9886 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a88030d578454489985b6252c4dab642 (127.23.186.195:34849), c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
W20260811 14:33:26.581673 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
W20260811 14:33:26.581825 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a88030d578454489985b6252c4dab642 (127.23.186.195:34849): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:26.581858 9401 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7473880090e544dda868033a4ce5370a; no voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94
I20260811 14:33:26.582060 9886 raft_consensus.cc:2726] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:33:27.041590 9887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:27.041148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:36293)
0811 14:33:27.041472 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:27.041531 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":89,"threads_started":1}
I20260811 14:33:27.105688 9888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:27.105356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:47215)
0811 14:33:27.105576 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:27.105633 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
I20260811 14:33:27.269536 9889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:27.269242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:43485)
0811 14:33:27.269424 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:27.269478 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":110,"threads_started":1}
I20260811 14:33:28.043286 9890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:28.042789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:34893)
0811 14:33:28.043160 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:28.043234 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":210,"threads_started":1}
I20260811 14:33:28.106901 9891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:28.106569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:45721)
0811 14:33:28.106783 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:28.106843 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":86,"threads_started":1}
I20260811 14:33:28.270706 9893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:28.270326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:60927)
0811 14:33:28.270544 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:28.270624 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":80,"threads_started":1}
I20260811 14:33:29.044576 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:29.044227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:46825)
0811 14:33:29.044457 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:29.044516 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
I20260811 14:33:29.107923 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:29.107634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:56963)
0811 14:33:29.107834 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:29.107869 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":134,"threads_started":1}
I20260811 14:33:29.272310 9896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:29.271859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:34033)
0811 14:33:29.272162 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:29.272247 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":124,"threads_started":1}
I20260811 14:33:29.531519 9897 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:29.531664 9897 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:29.531908 9897 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a88030d578454489985b6252c4dab642 (127.23.186.195:34849), c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
W20260811 14:33:29.532876 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a88030d578454489985b6252c4dab642 (127.23.186.195:34849): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
W20260811 14:33:29.533025 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:29.533066 9401 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7473880090e544dda868033a4ce5370a; no voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94
I20260811 14:33:29.533223 9897 raft_consensus.cc:2726] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:33:30.046049 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:30.045619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:52127)
0811 14:33:30.045907 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:30.045988 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":142,"threads_started":1}
I20260811 14:33:30.109314 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:30.108957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:56963)
0811 14:33:30.109193 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:30.109257 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
I20260811 14:33:30.273612 9901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:30.273180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:60739)
0811 14:33:30.273438 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:30.273529 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":87,"threads_started":1}
I20260811 14:33:31.047677 9902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:31.047230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:46299)
0811 14:33:31.047535 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:31.047598 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":114,"threads_started":1}
I20260811 14:33:31.110522 9903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:31.110146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:44285)
0811 14:33:31.110406 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:31.110448 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":98,"threads_started":1}
I20260811 14:33:31.275125 9904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:31.274640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:46923)
0811 14:33:31.274957 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:31.275046 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":90,"threads_started":1}
I20260811 14:33:32.049652 9905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:32.049198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:45775)
0811 14:33:32.049460 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:32.049548 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":123,"threads_started":1}
I20260811 14:33:32.111600 9906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:32.111261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:36023)
0811 14:33:32.111493 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:32.111545 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":79,"threads_started":1}
I20260811 14:33:32.276662 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:32.276285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:46069)
0811 14:33:32.276530 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:32.276591 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":113,"threads_started":1}
I20260811 14:33:33.051126 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:33.050660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:40663)
0811 14:33:33.050974 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:33.051058 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":108,"threads_started":1}
I20260811 14:33:33.113301 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:33.112823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:47291)
0811 14:33:33.113086 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:33.113213 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":138,"threads_started":1}
I20260811 14:33:33.278141 9910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:33.277773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:48071)
0811 14:33:33.278022 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:33.278085 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":106,"threads_started":1}
I20260811 14:33:33.325261 9911 raft_consensus.cc:482] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:33.325395 9911 raft_consensus.cc:504] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a88030d578454489985b6252c4dab642" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 34849 } } peers { permanent_uuid: "c0543a258afc4b59b7a2be4b0d950d94" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 33789 } } peers { permanent_uuid: "7473880090e544dda868033a4ce5370a" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 46117 } }
I20260811 14:33:33.325646 9911 leader_election.cc:288] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a88030d578454489985b6252c4dab642 (127.23.186.195:34849), c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789)
W20260811 14:33:33.326182 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a88030d578454489985b6252c4dab642 (127.23.186.195:34849): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
W20260811 14:33:33.326252 9401 leader_election.cc:341] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c0543a258afc4b59b7a2be4b0d950d94 (127.23.186.194:33789): Not found: Tablet not found: 3f7d946ed1754ce99d2a476ffd93c195
I20260811 14:33:33.326268 9401 leader_election.cc:302] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7473880090e544dda868033a4ce5370a; no voters: a88030d578454489985b6252c4dab642, c0543a258afc4b59b7a2be4b0d950d94
I20260811 14:33:33.326341 9911 raft_consensus.cc:2726] T 3f7d946ed1754ce99d2a476ffd93c195 P 7473880090e544dda868033a4ce5370a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:33:34.052513 9912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:34.052061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.193:45381)
0811 14:33:34.052356 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:34.052437 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":104,"threads_started":1}
I20260811 14:33:34.114532 9913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:34.114131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.194:56463)
0811 14:33:34.114378 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:34.114459 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
I20260811 14:33:34.269364 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 9345
I20260811 14:33:34.279457 9914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:34.279038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:35117 (local address 127.23.186.195:36429)
0811 14:33:34.279323 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:34.279384 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:35117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":207,"threads_started":1}
I20260811 14:33:34.281641 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 9535
I20260811 14:33:34.291648 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 9697
2026-08-11T14:33:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19695 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:33:34 dist-test-slave-cbsf krb5kdc[9917](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:33:34 dist-test-slave-cbsf krb5kdc[9917](info): set up 2 sockets
Aug 11 14:33:34 dist-test-slave-cbsf krb5kdc[9917](info): commencing operation
krb5kdc: starting...
W20260811 14:33:35.496028 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:33:35 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:33:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:33:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:33:35.661931 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38929
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:38929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:35.767710 9932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:35.768049 9932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:35.768093 9932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:35.769675 9932 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 14:33:35.769783 9932 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 14:33:35.769804 9932 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 14:33:35.769825 9932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:35.769848 9932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:35.769881 9932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:35.769903 9932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:35.769934 9932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:35.771989 9932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:38929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:35.772398 9932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:35.772764 9932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:35.779197 9943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:35.779213 9941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:35.779363 9939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:35.779388 9932 server_base.cc:1034] running on GCE node
I20260811 14:33:35.780076 9932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:35.780665 9932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:35.781909 9932 hybrid_clock.cc:648] HybridClock initialized: now 1786458815781897 us; error 44 us; skew 500 ppm
Aug 11 14:33:35 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458815, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:35.785660 9932 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:35.786273 9932 webserver.cc:460] Webserver started at http://127.23.186.254:44263/ using document root <none> and password file <none>
I20260811 14:33:35.786547 9932 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:35.786605 9932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:35.786741 9932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:35.787983 9932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "03ed8ac4bea64e79936331529deee5e5"
format_stamp: "Formatted at 2026-08-11 14:33:35 on dist-test-slave-cbsf"
server_key: "80d8779a0866c5cfa3dc0fa257a84641"
server_key_iv: "554dbe8b76a2bdb66864ab8040628fe0"
server_key_version: "encryptionkey@0"
I20260811 14:33:35.788335 9932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "03ed8ac4bea64e79936331529deee5e5"
format_stamp: "Formatted at 2026-08-11 14:33:35 on dist-test-slave-cbsf"
server_key: "80d8779a0866c5cfa3dc0fa257a84641"
server_key_iv: "554dbe8b76a2bdb66864ab8040628fe0"
server_key_version: "encryptionkey@0"
I20260811 14:33:35.790343 9932 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:33:35.791416 9951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:35.791870 9932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:35.791977 9932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "03ed8ac4bea64e79936331529deee5e5"
format_stamp: "Formatted at 2026-08-11 14:33:35 on dist-test-slave-cbsf"
server_key: "80d8779a0866c5cfa3dc0fa257a84641"
server_key_iv: "554dbe8b76a2bdb66864ab8040628fe0"
server_key_version: "encryptionkey@0"
I20260811 14:33:35.792129 9932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:35.811309 9932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:35.814622 9932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:35.814780 9932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:35.820194 9932 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:38929
I20260811 14:33:35.820236 10008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:38929 every 8 connection(s)
I20260811 14:33:35.820677 9932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:35.821637 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 9932
I20260811 14:33:35.821777 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:33:35.821995 10010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:35.822077 24299 external_mini_cluster.cc:1428] Setting key aaf25db0224cefe589f625887d826c6b
I20260811 14:33:35.826227 10010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap starting.
I20260811 14:33:35.827126 10010 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:35.827492 10010 log.cc:824] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:35.828361 10010 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: No bootstrap required, opened a new log
Aug 11 14:33:35 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458815, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:35.830637 10010 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:35.830866 10010 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:35.830940 10010 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ed8ac4bea64e79936331529deee5e5, State: Initialized, Role: FOLLOWER
I20260811 14:33:35.831130 10010 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:35.831241 10010 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:35.831291 10010 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:35.831346 10010 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:35.832059 10010 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:35.832217 10010 leader_election.cc:302] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ed8ac4bea64e79936331529deee5e5; no voters:
I20260811 14:33:35.832468 10010 leader_election.cc:288] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:33:35.832620 10015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:35.832858 10015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 LEADER]: Becoming Leader. State: Replica: 03ed8ac4bea64e79936331529deee5e5, State: Running, Role: LEADER
I20260811 14:33:35.832866 10010 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:35.833024 10015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:35.833493 10016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03ed8ac4bea64e79936331529deee5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } } }
I20260811 14:33:35.833479 10017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ed8ac4bea64e79936331529deee5e5. Latest consensus state: current_term: 1 leader_uuid: "03ed8ac4bea64e79936331529deee5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } } }
I20260811 14:33:35.833609 10016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:35.833613 10017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:35.834488 10021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:35.835235 10021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:35.835691 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:35.822990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.23.186.254:38929)
0811 14:33:35.823233 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:35.823244 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:35.823282 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:33:35.823864 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:35.823876 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:35.823958 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:35.824161 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:35.825148 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:35.826662 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:35.827708 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:35.828219 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:35.828373 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:35.831154 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:35.831159 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:35.831160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:35.831175 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:35.833537 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:35.834130 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:35.834134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:35.834134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:35.834272 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:35.834545 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:35.834546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:35.834547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:35.834962 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:35.835193 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:35.835275 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 14:33:35.835443 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":112,"threads_started":1}
I20260811 14:33:35.837618 10021 catalog_manager.cc:1347] Generated new cluster ID: d00b52b83e1240089cc5c6853d69a92d
I20260811 14:33:35.837713 10021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:35.854295 10021 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:33:35.855206 10021 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:35.863479 10021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Generated new TSK 0
I20260811 14:33:35.863890 10021 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:33:35.914644 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38929
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:36.050393 10040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:36.050763 10040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:36.050814 10040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:36.052539 10040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:36.052647 10040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:36.052708 10040 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:36.056283 10040 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10040
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:36.056821 10040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:36.057168 10040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:36.063380 10048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:36.063614 10040 server_base.cc:1034] running on GCE node
W20260811 14:33:36.063402 10050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:36.063400 10047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:36.064232 10040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:36.064581 10040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:36.065848 10040 hybrid_clock.cc:648] HybridClock initialized: now 1786458816065833 us; error 63 us; skew 500 ppm
Aug 11 14:33:36 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458816, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:36.071064 10040 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:36.072122 10040 webserver.cc:460] Webserver started at http://127.23.186.193:45259/ using document root <none> and password file <none>
I20260811 14:33:36.072547 10040 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:36.072613 10040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:36.072853 10040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:36.075169 10040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "b0a233dde4dd408281418532370591ed"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "e01f5ffb258f35f33f7411e0783e8924"
server_key_iv: "531abfaa13aaa72a1c343cc43d51bd23"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.075898 10040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "b0a233dde4dd408281418532370591ed"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "e01f5ffb258f35f33f7411e0783e8924"
server_key_iv: "531abfaa13aaa72a1c343cc43d51bd23"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.078172 10040 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260811 14:33:36.079313 10058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.079631 10040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:36.079743 10040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "b0a233dde4dd408281418532370591ed"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "e01f5ffb258f35f33f7411e0783e8924"
server_key_iv: "531abfaa13aaa72a1c343cc43d51bd23"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.079891 10040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:36.096387 10040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:36.100037 10040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:36.100234 10040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:36.100564 10040 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:36.101147 10040 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:36.101197 10040 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.101230 10040 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:36.101245 10040 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.111663 10040 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:44989
I20260811 14:33:36.111728 10192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:44989 every 8 connection(s)
I20260811 14:33:36.112197 10040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:36.119836 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10040
I20260811 14:33:36.119987 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:33:36.120239 24299 external_mini_cluster.cc:1428] Setting key ca3575d10fa51fd9155e3bca5214a30e
Aug 11 14:33:36 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458816, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:36.126910 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.113560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:47361 (local address 127.23.186.254:38929)
0811 14:33:36.113630 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.113632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.113925 (+ 293us) server_negotiation.cc:407] Connection header received
0811 14:33:36.114803 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.114806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.114823 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.114936 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:36.116431 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.117169 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.118779 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.119003 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.119108 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.122505 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:36.122508 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:36.122508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:36.122519 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:36.124996 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.125576 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.125578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.125578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.125662 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.125972 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.125973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.125974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.126162 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:36.126244 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.126649 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.126778 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:33:36.127161 10195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.113458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:47361)
0811 14:33:36.113694 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.113752 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.114626 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.115066 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.115072 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.115954 (+ 882us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:36.116271 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.116278 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.117340 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.117342 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.118606 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.118612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.119170 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.119172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.119372 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.120407 (+ 1035us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:36.120421 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:36.122375 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:36.125131 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:36.125145 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:36.125149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:36.125466 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:36.125746 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:36.125747 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:36.125747 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:36.125886 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:36.126298 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:36.126307 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:36.126470 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.126591 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.126782 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":132,"threads_started":1}
I20260811 14:33:36.128075 10193 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38929
I20260811 14:33:36.128321 10193 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:36.128968 10193 heartbeater.cc:511] Master 127.23.186.254:38929 requested a full tablet report, sending...
I20260811 14:33:36.129949 9971 ts_manager.cc:194] Registered new tserver with Master: b0a233dde4dd408281418532370591ed (127.23.186.193:44989)
I20260811 14:33:36.130681 9971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:47361
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:33:36.161607 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38929
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:36.273125 10199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:36.273404 10199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:36.273435 10199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:36.274835 10199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:36.274906 10199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:36.274946 10199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:36.276680 10199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:36.277076 10199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:36.277361 10199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:36.283900 10206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:36.283907 10208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:36.283907 10205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:36.284605 10199 server_base.cc:1034] running on GCE node
I20260811 14:33:36.284937 10199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:36.285315 10199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:36.286578 10199 hybrid_clock.cc:648] HybridClock initialized: now 1786458816286496 us; error 105 us; skew 500 ppm
Aug 11 14:33:36 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458816, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:36.290114 10199 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:36.290798 10199 webserver.cc:460] Webserver started at http://127.23.186.194:35649/ using document root <none> and password file <none>
I20260811 14:33:36.291119 10199 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:36.291208 10199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:36.291419 10199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:36.292788 10199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "7c809573d8a6460498758e2ecaad1075"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "c1d430a566923d77f95853cae7c4d7d8"
server_key_iv: "db0b173a45a509facd1d4ab3c789b182"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.293193 10199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "7c809573d8a6460498758e2ecaad1075"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "c1d430a566923d77f95853cae7c4d7d8"
server_key_iv: "db0b173a45a509facd1d4ab3c789b182"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.295493 10199 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 14:33:36.296950 10218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.297515 10199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:36.297662 10199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "7c809573d8a6460498758e2ecaad1075"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "c1d430a566923d77f95853cae7c4d7d8"
server_key_iv: "db0b173a45a509facd1d4ab3c789b182"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.297760 10199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:36.337061 10199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:36.340919 10199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:36.341135 10199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:36.341576 10199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:36.342097 10199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:36.342147 10199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.342190 10199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:36.342206 10199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.352062 10199 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:42857
I20260811 14:33:36.352116 10353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:42857 every 8 connection(s)
I20260811 14:33:36.352641 10199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:33:36 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458816, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:36.362892 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10199
I20260811 14:33:36.363250 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:33:36.363642 24299 external_mini_cluster.cc:1428] Setting key ebfe1a8f4cb8175dd37279e0cdeefdf2
I20260811 14:33:36.367100 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.353829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:52881 (local address 127.23.186.254:38929)
0811 14:33:36.354036 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.354039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.354209 (+ 170us) server_negotiation.cc:407] Connection header received
0811 14:33:36.354853 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.354855 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.354877 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.354990 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:36.356299 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.357030 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.358627 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.358893 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.359017 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.362691 (+ 3674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:36.362695 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:36.362695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:36.362710 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:36.365172 (+ 2462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.365857 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.365858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.365859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.365944 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.366265 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.366266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.366267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.366421 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:36.366507 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.366867 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.366993 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 14:33:36.367360 10356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.353708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:52881)
0811 14:33:36.353998 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.354050 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.354615 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.355115 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.355120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.355850 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:36.356118 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.356123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.357262 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.357265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.358427 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.358432 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.359103 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.359105 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.359325 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.360102 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:36.360117 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:36.362416 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:36.365336 (+ 2920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:36.365357 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:36.365361 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:36.365743 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:36.366034 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:36.366034 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:36.366035 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:36.366174 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:36.366508 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:36.366519 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:36.366703 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.366833 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.367053 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":160,"threads_started":1}
I20260811 14:33:36.368016 10354 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38929
I20260811 14:33:36.368259 10354 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:36.368608 10354 heartbeater.cc:511] Master 127.23.186.254:38929 requested a full tablet report, sending...
I20260811 14:33:36.369491 9971 ts_manager.cc:194] Registered new tserver with Master: 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857)
I20260811 14:33:36.370121 9971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:52881
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:33:36.406383 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38929
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:36.509405 10361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:36.509683 10361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:36.509713 10361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:36.511118 10361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:36.511185 10361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:36.511217 10361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:36.513341 10361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:36.513865 10361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:36.514237 10361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:36.521277 10367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:36.521333 10368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:36.521256 10370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:36.521695 10361 server_base.cc:1034] running on GCE node
I20260811 14:33:36.522059 10361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:36.522450 10361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:36.523715 10361 hybrid_clock.cc:648] HybridClock initialized: now 1786458816523673 us; error 54 us; skew 500 ppm
Aug 11 14:33:36 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458816, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:36.527266 10361 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:36.527894 10361 webserver.cc:460] Webserver started at http://127.23.186.195:39885/ using document root <none> and password file <none>
I20260811 14:33:36.528187 10361 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:36.528270 10361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:36.528448 10361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:33:36.529806 10361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "58a1f1fb5fe944d28f402fe090706160"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "a1d04c39ef16eef3d9265ad8dfed715a"
server_key_iv: "a58a998ef0d1529319ec692face06c42"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.530269 10361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "58a1f1fb5fe944d28f402fe090706160"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "a1d04c39ef16eef3d9265ad8dfed715a"
server_key_iv: "a58a998ef0d1529319ec692face06c42"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.532069 10361 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 14:33:36.533509 10377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.534044 10361 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:36.534237 10361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "58a1f1fb5fe944d28f402fe090706160"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "a1d04c39ef16eef3d9265ad8dfed715a"
server_key_iv: "a58a998ef0d1529319ec692face06c42"
server_key_version: "encryptionkey@0"
I20260811 14:33:36.534415 10361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:36.553617 10361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:36.557655 10361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:36.557914 10361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:36.558296 10361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:36.558766 10361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:36.558849 10361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.558897 10361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:36.558971 10361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:36.566672 10361 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:45617
I20260811 14:33:36.566731 10517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:45617 every 8 connection(s)
I20260811 14:33:36.567320 10361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:33:36.575575 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10361
I20260811 14:33:36.575762 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:33:36.576040 24299 external_mini_cluster.cc:1428] Setting key 8bfa6613c53cc4d9f30c70f2f5c75b70
Aug 11 14:33:36 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458816, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:36.583973 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.568984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:53997 (local address 127.23.186.254:38929)
0811 14:33:36.569075 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.569078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.569170 (+ 92us) server_negotiation.cc:407] Connection header received
0811 14:33:36.570124 (+ 954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.570127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.570141 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.570216 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:36.571820 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.572383 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.574713 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.574978 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.575104 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.578487 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:36.578489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:36.578490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:36.578500 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:36.580971 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.581891 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.581893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.581893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.581985 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.582726 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.582729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.582730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.582961 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:36.583041 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.583740 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.583896 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 14:33:36.584156 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.568776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:53997)
0811 14:33:36.569008 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.569066 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.569892 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.570323 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.570328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.571168 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:36.571658 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.571670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.572580 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.572582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.574539 (+ 1957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.574546 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.575171 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.575173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.575328 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.576078 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:36.576112 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:36.578355 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:36.581239 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:36.581264 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:36.581270 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:36.581750 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:36.582073 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:36.582075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:36.582075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:36.582249 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:36.583125 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:36.583144 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:36.583492 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.583656 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.583878 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":130,"threads_started":1}
I20260811 14:33:36.584877 10519 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38929
I20260811 14:33:36.584991 10519 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:36.585177 10519 heartbeater.cc:511] Master 127.23.186.254:38929 requested a full tablet report, sending...
I20260811 14:33:36.585670 9971 ts_manager.cc:194] Registered new tserver with Master: 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
I20260811 14:33:36.586066 9971 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:53997
I20260811 14:33:36.589730 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:33:36.600260 10013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.591647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.23.186.254:38929)
0811 14:33:36.591699 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.591702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.591855 (+ 153us) server_negotiation.cc:407] Connection header received
0811 14:33:36.591891 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.591893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.591904 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.591973 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:36.592662 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.593289 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.594127 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.594326 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.594392 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.596580 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:36.596583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:36.596583 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:36.596595 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:36.598351 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.598993 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.598994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.598994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.599085 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:36.599411 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:36.599412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:36.599412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:36.599789 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:36.599897 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.600097 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.600185 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:33:36.603279 9971 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:33:36.605826 9971 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:33:36.614534 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.609630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56052 (local address 127.23.186.194:42857)
0811 14:33:36.609843 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.609849 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.609953 (+ 104us) server_negotiation.cc:407] Connection header received
0811 14:33:36.610004 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.610006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.610022 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.610116 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:36.610513 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.611592 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.613147 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.613966 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.614050 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.614121 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.614369 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.614426 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":99,"threads_started":1}
I20260811 14:33:36.614521 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.609285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:45617 (local address 127.0.0.1:59610)
0811 14:33:36.609634 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.609683 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.609768 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.610330 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.610331 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.610333 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:36.610407 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.610409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.611767 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.611769 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.613375 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.613377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.614123 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.614124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.614225 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.614241 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.614353 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.614435 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":139,"thread_start_us":108,"threads_started":1}
I20260811 14:33:36.614573 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.609553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.23.186.193:44989)
0811 14:33:36.609914 (+ 361us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.609918 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.609930 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:33:36.609983 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.609985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.610107 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.610219 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:36.610410 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.611655 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.613376 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.614098 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.614168 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.614217 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.614416 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.614474 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":77,"threads_started":1}
I20260811 14:33:36.614827 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.609528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.23.186.195:45617)
0811 14:33:36.610073 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.610100 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.610112 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:33:36.610156 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.610159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.610176 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.610254 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:36.610494 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.611641 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.613483 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.614013 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.614127 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.614188 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.614721 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.614764 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":116,"threads_started":1}
I20260811 14:33:36.614841 10530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.609285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44989 (local address 127.0.0.1:52618)
0811 14:33:36.609732 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.609746 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.609802 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.610234 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.610238 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.610241 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:36.610323 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.610326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.611780 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.611781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.613270 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.613274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.614251 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.614252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.614313 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.614317 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.614363 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.614754 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":448,"thread_start_us":118,"threads_started":1}
I20260811 14:33:36.615633 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.609298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:42857 (local address 127.0.0.1:56052)
0811 14:33:36.609871 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.609879 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.609943 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.610348 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.610349 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.610350 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:36.610415 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.610417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.611764 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.611765 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.613035 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.613038 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.614162 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.614163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.614231 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.614257 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.614324 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.615582 (+ 1258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":69,"thread_start_us":147,"threads_started":1}
I20260811 14:33:36.615754 10115 tablet_service.cc:1467] Processing CreateTablet for tablet 807385a1ad2947b8b64048aeced6640a (DEFAULT_TABLE table=test-table [id=77456f2d19a643fe8be6f6961a0eb873]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:36.615918 10441 tablet_service.cc:1467] Processing CreateTablet for tablet 807385a1ad2947b8b64048aeced6640a (DEFAULT_TABLE table=test-table [id=77456f2d19a643fe8be6f6961a0eb873]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:36.616216 10115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807385a1ad2947b8b64048aeced6640a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:36.616216 10441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807385a1ad2947b8b64048aeced6640a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:36.616329 10279 tablet_service.cc:1467] Processing CreateTablet for tablet 807385a1ad2947b8b64048aeced6640a (DEFAULT_TABLE table=test-table [id=77456f2d19a643fe8be6f6961a0eb873]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:36.616598 10279 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 807385a1ad2947b8b64048aeced6640a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:36.620399 10537 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Bootstrap starting.
I20260811 14:33:36.620539 10539 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Bootstrap starting.
I20260811 14:33:36.621790 10539 tablet_bootstrap.cc:654] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:36.622182 10537 tablet_bootstrap.cc:654] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:36.622623 10539 log.cc:824] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:36.622852 10537 log.cc:824] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:36.623462 10540 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Bootstrap starting.
I20260811 14:33:36.624063 10539 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: No bootstrap required, opened a new log
I20260811 14:33:36.624171 10539 ts_tablet_manager.cc:1397] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 14:33:36.624312 10537 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: No bootstrap required, opened a new log
I20260811 14:33:36.624405 10537 ts_tablet_manager.cc:1397] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260811 14:33:36.624516 10540 tablet_bootstrap.cc:654] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:36.625272 10540 log.cc:824] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:36.626314 10539 raft_consensus.cc:348] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.626540 10539 raft_consensus.cc:374] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:36.626585 10539 raft_consensus.cc:729] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a233dde4dd408281418532370591ed, State: Initialized, Role: FOLLOWER
I20260811 14:33:36.626595 10540 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: No bootstrap required, opened a new log
I20260811 14:33:36.626683 10540 ts_tablet_manager.cc:1397] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:33:36.626744 10539 consensus_queue.cc:260] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.627168 10539 ts_tablet_manager.cc:1428] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:33:36.627285 10193 heartbeater.cc:503] Master 127.23.186.254:38929 was elected leader, sending a full tablet report...
I20260811 14:33:36.627272 10537 raft_consensus.cc:348] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.627543 10537 raft_consensus.cc:374] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:36.627586 10537 raft_consensus.cc:729] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c809573d8a6460498758e2ecaad1075, State: Initialized, Role: FOLLOWER
I20260811 14:33:36.627743 10537 consensus_queue.cc:260] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.628171 10537 ts_tablet_manager.cc:1428] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260811 14:33:36.628175 10354 heartbeater.cc:503] Master 127.23.186.254:38929 was elected leader, sending a full tablet report...
I20260811 14:33:36.628988 10540 raft_consensus.cc:348] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.629256 10540 raft_consensus.cc:374] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:36.629309 10540 raft_consensus.cc:729] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a1f1fb5fe944d28f402fe090706160, State: Initialized, Role: FOLLOWER
I20260811 14:33:36.629460 10540 consensus_queue.cc:260] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.629899 10540 ts_tablet_manager.cc:1428] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:33:36.629910 10519 heartbeater.cc:503] Master 127.23.186.254:38929 was elected leader, sending a full tablet report...
W20260811 14:33:36.818284 10520 tablet.cc:2369] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:36.853885 10355 tablet.cc:2369] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:36.863512 10194 tablet.cc:2369] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:36.960357 10545 raft_consensus.cc:482] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:36.960592 10545 raft_consensus.cc:504] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.961232 10545 leader_election.cc:288] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b0a233dde4dd408281418532370591ed (127.23.186.193:44989), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
I20260811 14:33:36.965056 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.961894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:41941 (local address 127.23.186.195:45617)
0811 14:33:36.961968 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.961970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.962060 (+ 90us) server_negotiation.cc:407] Connection header received
0811 14:33:36.962160 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.962161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.962173 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.962266 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:36.962498 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.963049 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.964185 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.964602 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.964708 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.964740 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.964928 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.964974 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 14:33:36.965075 10549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.961631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:45617 (local address 127.23.186.194:41941)
0811 14:33:36.961919 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.961939 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.962072 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.962280 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.962281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.962285 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:36.962398 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.962401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.963158 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.963160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.964055 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.964059 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.964740 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.964742 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.964829 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.964833 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.964913 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.964967 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":134,"threads_started":1}
I20260811 14:33:36.965394 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.961904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:33761 (local address 127.23.186.193:44989)
0811 14:33:36.961969 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:36.961972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:36.962105 (+ 133us) server_negotiation.cc:407] Connection header received
0811 14:33:36.962247 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:36.962248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:36.962263 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:36.962348 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:36.962593 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.963283 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.964380 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.964858 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.964957 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.964966 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:36.965250 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 14:33:36.965312 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:33:36.965646 10548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:36.961631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44989 (local address 127.23.186.194:33761)
0811 14:33:36.962000 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:36.962012 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:36.962127 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:36.962357 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:36.962358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:36.962362 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:36.962505 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.962508 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.963457 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.963458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.964249 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:36.964251 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:36.964962 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:36.964964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:36.965119 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:36.965123 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:33:36.965516 (+ 393us) client_negotiation.cc:240] Negotiation successful
0811 14:33:36.965568 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":37,"thread_start_us":119,"threads_started":1}
I20260811 14:33:36.965698 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807385a1ad2947b8b64048aeced6640a" candidate_uuid: "7c809573d8a6460498758e2ecaad1075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a1f1fb5fe944d28f402fe090706160" is_pre_election: true
I20260811 14:33:36.965966 10463 raft_consensus.cc:2454] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c809573d8a6460498758e2ecaad1075 in term 0.
I20260811 14:33:36.966073 10138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807385a1ad2947b8b64048aeced6640a" candidate_uuid: "7c809573d8a6460498758e2ecaad1075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a233dde4dd408281418532370591ed" is_pre_election: true
I20260811 14:33:36.966230 10138 raft_consensus.cc:2454] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c809573d8a6460498758e2ecaad1075 in term 0.
I20260811 14:33:36.966270 10220 leader_election.cc:302] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a1f1fb5fe944d28f402fe090706160, 7c809573d8a6460498758e2ecaad1075; no voters:
I20260811 14:33:36.966440 10545 raft_consensus.cc:2781] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:33:36.966543 10545 raft_consensus.cc:482] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:33:36.966580 10545 raft_consensus.cc:3037] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:36.967455 10545 raft_consensus.cc:504] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.967658 10545 leader_election.cc:288] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [CANDIDATE]: Term 1 election: Requested vote from peers b0a233dde4dd408281418532370591ed (127.23.186.193:44989), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
I20260811 14:33:36.967856 10138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807385a1ad2947b8b64048aeced6640a" candidate_uuid: "7c809573d8a6460498758e2ecaad1075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b0a233dde4dd408281418532370591ed"
I20260811 14:33:36.967978 10138 raft_consensus.cc:3037] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:36.968974 10138 raft_consensus.cc:2454] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c809573d8a6460498758e2ecaad1075 in term 1.
I20260811 14:33:36.968981 10463 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "807385a1ad2947b8b64048aeced6640a" candidate_uuid: "7c809573d8a6460498758e2ecaad1075" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a1f1fb5fe944d28f402fe090706160"
I20260811 14:33:36.969077 10463 raft_consensus.cc:3037] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:36.969239 10222 leader_election.cc:302] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c809573d8a6460498758e2ecaad1075, b0a233dde4dd408281418532370591ed; no voters:
I20260811 14:33:36.969383 10545 raft_consensus.cc:2781] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:36.969617 10545 raft_consensus.cc:686] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 1 LEADER]: Becoming Leader. State: Replica: 7c809573d8a6460498758e2ecaad1075, State: Running, Role: LEADER
I20260811 14:33:36.969760 10545 consensus_queue.cc:237] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:36.970113 10463 raft_consensus.cc:2454] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c809573d8a6460498758e2ecaad1075 in term 1.
I20260811 14:33:36.970897 9971 catalog_manager.cc:5495] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c809573d8a6460498758e2ecaad1075 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "7c809573d8a6460498758e2ecaad1075" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } health_report { overall_health: HEALTHY } } }
I20260811 14:33:37.066174 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:37.063183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56068 (local address 127.23.186.194:42857)
0811 14:33:37.063298 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:37.063300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:37.063310 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:33:37.063357 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:37.063358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:37.063383 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:37.063467 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:37.063749 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:37.064272 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:37.065205 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:37.065411 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:37.065478 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:37.065705 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:37.065902 (+ 197us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:33:37.065996 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:37.066051 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 14:33:37.066082 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 14:33:37.068786 10138 raft_consensus.cc:1261] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Refusing update from remote peer 7c809573d8a6460498758e2ecaad1075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:37.068812 10463 raft_consensus.cc:1261] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Refusing update from remote peer 7c809573d8a6460498758e2ecaad1075: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:37.069268 10550 consensus_queue.cc:1035] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:37.069449 10545 consensus_queue.cc:1035] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:37.072413 10553 mvcc.cc:205] Tried to move back new op lower bound from 7317335314709585920 to 7317335314308968448. Current Snapshot: MvccSnapshot[applied={T|T < 7317335314709585920}]
I20260811 14:33:37.075755 9968 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:37.075937 9968 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:37.077514 9968 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Sending DeleteTablet for 3 replicas of tablet 807385a1ad2947b8b64048aeced6640a
I20260811 14:33:37.078227 10115 tablet_service.cc:1514] Processing DeleteTablet for tablet 807385a1ad2947b8b64048aeced6640a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:37 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:52618
I20260811 14:33:37.078269 10279 tablet_service.cc:1514] Processing DeleteTablet for tablet 807385a1ad2947b8b64048aeced6640a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:37 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:56052
I20260811 14:33:37.078286 10441 tablet_service.cc:1514] Processing DeleteTablet for tablet 807385a1ad2947b8b64048aeced6640a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:37 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:59610
I20260811 14:33:37.078851 10566 tablet_replica.cc:330] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: stopping tablet replica
I20260811 14:33:37.078899 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10040
I20260811 14:33:37.079030 10566 raft_consensus.cc:2229] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:37.079197 10566 raft_consensus.cc:2258] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:37.081209 10567 tablet_replica.cc:330] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: stopping tablet replica
I20260811 14:33:37.081635 10567 raft_consensus.cc:2229] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:33:37.081921 10567 raft_consensus.cc:2258] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:37.083313 10567 ts_tablet_manager.cc:1905] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:37.086068 10566 ts_tablet_manager.cc:1905] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:37.086777 10567 ts_tablet_manager.cc:1918] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:37.086880 10567 log.cc:1196] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/807385a1ad2947b8b64048aeced6640a
I20260811 14:33:37.087225 10567 ts_tablet_manager.cc:1939] T 807385a1ad2947b8b64048aeced6640a P 7c809573d8a6460498758e2ecaad1075: Deleting consensus metadata
I20260811 14:33:37.087875 9953 catalog_manager.cc:4841] TS 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857): tablet 807385a1ad2947b8b64048aeced6640a (table test-table [id=77456f2d19a643fe8be6f6961a0eb873]) successfully deleted
I20260811 14:33:37.088116 10566 ts_tablet_manager.cc:1918] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:37.088205 10566 log.cc:1196] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/807385a1ad2947b8b64048aeced6640a
I20260811 14:33:37.088490 10566 ts_tablet_manager.cc:1939] T 807385a1ad2947b8b64048aeced6640a P 58a1f1fb5fe944d28f402fe090706160: Deleting consensus metadata
I20260811 14:33:37.089026 9954 catalog_manager.cc:4841] TS 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617): tablet 807385a1ad2947b8b64048aeced6640a (table test-table [id=77456f2d19a643fe8be6f6961a0eb873]) successfully deleted
W20260811 14:33:37.093761 9956 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:44989 (error 108)
I20260811 14:33:37.094178 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10199
I20260811 14:33:37.096393 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:37.094347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44989 (local address 127.0.0.1:52630)
0811 14:33:37.096232 (+ 1885us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:37.096330 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1841}
W20260811 14:33:37.101366 9956 catalog_manager.cc:4584] TS b0a233dde4dd408281418532370591ed (127.23.186.193:44989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 807385a1ad2947b8b64048aeced6640a: Network error: Client connection negotiation failed: client connection to 127.23.186.193:44989: connect: Connection refused (error 111)
I20260811 14:33:37.108428 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10361
I20260811 14:33:37.123035 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 9932
I20260811 14:33:57.133958 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38929
--webserver_interface=127.23.186.254
--webserver_port=44263
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:38929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:33:57.223258 10569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:57.223536 10569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:57.223582 10569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:57.224879 10569 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 14:33:57.224954 10569 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 14:33:57.224979 10569 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 14:33:57.224999 10569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:33:57.225032 10569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:57.225066 10569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:57.225095 10569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:33:57.225119 10569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:33:57.226773 10569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:38929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:57.227108 10569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:57.227371 10569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:57.237632 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.237653 10575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.237660 10578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:57.237952 10569 server_base.cc:1034] running on GCE node
I20260811 14:33:57.238406 10569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:57.238732 10569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:57.239898 10569 hybrid_clock.cc:648] HybridClock initialized: now 1786458837239884 us; error 30 us; skew 500 ppm
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:57.242933 10569 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:33:57.243512 10569 webserver.cc:460] Webserver started at http://127.23.186.254:44263/ using document root <none> and password file <none>
I20260811 14:33:57.243789 10569 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:57.243891 10569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:57.245774 10569 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:33:57.246690 10588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.247078 10569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:57.247217 10569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "03ed8ac4bea64e79936331529deee5e5"
format_stamp: "Formatted at 2026-08-11 14:33:35 on dist-test-slave-cbsf"
server_key: "80d8779a0866c5cfa3dc0fa257a84641"
server_key_iv: "554dbe8b76a2bdb66864ab8040628fe0"
server_key_version: "encryptionkey@0"
I20260811 14:33:57.247699 10569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:57.266827 10569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:57.269404 10569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:57.269599 10569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:57.274636 10569 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:38929
I20260811 14:33:57.274672 10646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:38929 every 8 connection(s)
I20260811 14:33:57.275121 10569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:33:57.276809 10647 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:33:57.277601 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap starting.
I20260811 14:33:57.281548 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10569
I20260811 14:33:57.281881 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:44989
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:38929
--webserver_port=45259
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:57.286798 10647 log.cc:824] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:57.289032 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:57.289386 10647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap complete.
I20260811 14:33:57.292096 10647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:57.292371 10647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ed8ac4bea64e79936331529deee5e5, State: Initialized, Role: FOLLOWER
I20260811 14:33:57.292593 10647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:57.292694 10647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:33:57.292778 10647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:33:57.292830 10647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 14:33:57.293951 10647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:57.294088 10647 leader_election.cc:302] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ed8ac4bea64e79936331529deee5e5; no voters:
I20260811 14:33:57.294292 10647 leader_election.cc:288] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 14:33:57.294374 10651 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 14:33:57.294548 10651 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 LEADER]: Becoming Leader. State: Replica: 03ed8ac4bea64e79936331529deee5e5, State: Running, Role: LEADER
I20260811 14:33:57.294608 10647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:33:57.294653 10651 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:33:57.294945 10651 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ed8ac4bea64e79936331529deee5e5. Latest consensus state: current_term: 2 leader_uuid: "03ed8ac4bea64e79936331529deee5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } } }
I20260811 14:33:57.295017 10651 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:57.295156 10652 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "03ed8ac4bea64e79936331529deee5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } } }
I20260811 14:33:57.295208 10652 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:33:57.295310 10656 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:33:57.296182 10656 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:33:57.296386 10656 catalog_manager.cc:1259] Loaded cluster ID: d00b52b83e1240089cc5c6853d69a92d
I20260811 14:33:57.296649 10656 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:33:57.297904 10656 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:33:57.298111 10656 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Loaded TSK: 0
I20260811 14:33:57.317027 10656 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Generated new TSK 1
I20260811 14:33:57.317188 10656 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:33:57.371984 10649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:57.372215 10649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:57.372242 10649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:57.373551 10649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:57.373605 10649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:57.373636 10649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:33:57.375233 10649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:44989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=45259
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:57.375527 10649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:57.375805 10649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:57.381330 10680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.381321 10681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.381330 10683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:57.381588 10649 server_base.cc:1034] running on GCE node
I20260811 14:33:57.381855 10649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:57.382169 10649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:57.383330 10649 hybrid_clock.cc:648] HybridClock initialized: now 1786458837383316 us; error 34 us; skew 500 ppm
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:57.386224 10649 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:33:57.386693 10649 webserver.cc:460] Webserver started at http://127.23.186.193:45259/ using document root <none> and password file <none>
I20260811 14:33:57.387077 10649 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:57.387185 10649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:57.388849 10649 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:33:57.389729 10690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.389977 10649 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:57.390106 10649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "b0a233dde4dd408281418532370591ed"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "e01f5ffb258f35f33f7411e0783e8924"
server_key_iv: "531abfaa13aaa72a1c343cc43d51bd23"
server_key_version: "encryptionkey@0"
I20260811 14:33:57.390595 10649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:57.401062 10649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:57.403563 10649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:57.403751 10649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:57.404021 10649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:57.404503 10698 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:33:57.406713 10649 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:33:57.406793 10649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:33:57.406847 10649 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:33:57.407546 10649 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:33:57.407603 10649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 14:33:57.407680 10698 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Bootstrap starting.
I20260811 14:33:57.415439 10649 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:44989
I20260811 14:33:57.415881 10649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:33:57.418802 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10649
I20260811 14:33:57.419246 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:42857
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:38929
--webserver_port=35649
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:57.420073 10698 log.cc:824] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:57.421380 10818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:44989 every 8 connection(s)
I20260811 14:33:57.423048 10698 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:33:57.423409 10698 tablet_bootstrap.cc:492] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Bootstrap complete.
I20260811 14:33:57.424793 10698 ts_tablet_manager.cc:1397] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.016s
I20260811 14:33:57.426608 10698 raft_consensus.cc:348] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:57.426997 10698 raft_consensus.cc:729] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a233dde4dd408281418532370591ed, State: Initialized, Role: FOLLOWER
I20260811 14:33:57.427182 10698 consensus_queue.cc:260] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } }
I20260811 14:33:57.427523 10698 ts_tablet_manager.cc:1428] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:57.432621 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.416817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:58497 (local address 127.23.186.254:38929)
0811 14:33:57.419260 (+ 2443us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.419272 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.419557 (+ 285us) server_negotiation.cc:407] Connection header received
0811 14:33:57.419712 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.419719 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.419779 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.421166 (+ 1387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:57.422178 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.423593 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.424720 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.425223 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.425355 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.428369 (+ 3014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:57.428373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:57.428374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:57.428396 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:57.430638 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.431163 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.431166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.431166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.431279 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.431547 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.431548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.431548 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.431829 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:57.431882 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.432250 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.432406 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2365,"thread_start_us":96,"threads_started":1}
I20260811 14:33:57.433497 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.416679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:58497)
0811 14:33:57.416921 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:57.416977 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:57.417474 (+ 497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:57.421288 (+ 3814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:57.421293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:57.421704 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:57.422015 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.422024 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.423719 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.423720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.424572 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.424574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.425348 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.425349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.425509 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.426171 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:57.426195 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:57.428201 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:57.430744 (+ 2543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.430753 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.430756 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.431060 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.431347 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.431347 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.431348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.431477 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.431928 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:57.431934 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:57.432078 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 14:33:57.433024 (+ 946us) client_negotiation.cc:240] Negotiation successful
0811 14:33:57.433251 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260811 14:33:57.434578 10819 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38929
I20260811 14:33:57.434675 10819 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:57.434859 10819 heartbeater.cc:511] Master 127.23.186.254:38929 requested a full tablet report, sending...
I20260811 14:33:57.435669 10609 ts_manager.cc:194] Registered new tserver with Master: b0a233dde4dd408281418532370591ed (127.23.186.193:44989)
I20260811 14:33:57.436977 10609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:58497
I20260811 14:33:57.438040 10819 heartbeater.cc:503] Master 127.23.186.254:38929 was elected leader, sending a full tablet report...
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.193@KRBTEST.COM
I20260811 14:33:57.447944 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.436316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44989 (local address 127.0.0.1:57468)
0811 14:33:57.436817 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:57.436854 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:57.436922 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:57.437461 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:57.437467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:57.437807 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:57.437906 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.437908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.439543 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.439544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.440514 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.440516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.441189 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.441190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.441294 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.441873 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:57.441879 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:57.443785 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:57.446033 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.446040 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.446043 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.446307 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.446533 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.446534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.446534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.447041 (+ 507us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.447485 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:57.447494 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:57.447561 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 14:33:57.447658 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 14:33:57.447816 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":146,"threads_started":1}
I20260811 14:33:57.448570 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.436625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57468 (local address 127.23.186.193:44989)
0811 14:33:57.437055 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.437060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.437069 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:57.437103 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.437105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.437134 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.437194 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:57.438009 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.439446 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.440613 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.441081 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.441158 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.443884 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:57.443899 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:57.443900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:57.443908 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:57.445923 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.446397 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.446400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.446401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.446464 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.447125 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.447127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.447127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.447364 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:57.447451 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.448310 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.448439 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":104,"threads_started":1}
I20260811 14:33:57.449019 10745 tablet_service.cc:1514] Processing DeleteTablet for tablet 807385a1ad2947b8b64048aeced6640a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:37 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:57468
I20260811 14:33:57.450109 10831 tablet_replica.cc:330] stopping tablet replica
I20260811 14:33:57.450232 10831 raft_consensus.cc:2229] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:57.450306 10831 raft_consensus.cc:2258] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:57.450728 10831 ts_tablet_manager.cc:1905] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:57.453423 10831 ts_tablet_manager.cc:1918] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:57.453512 10831 log.cc:1196] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal/wals/807385a1ad2947b8b64048aeced6640a
I20260811 14:33:57.453752 10831 ts_tablet_manager.cc:1939] T 807385a1ad2947b8b64048aeced6640a P b0a233dde4dd408281418532370591ed: Deleting consensus metadata
I20260811 14:33:57.454391 10594 catalog_manager.cc:4841] TS b0a233dde4dd408281418532370591ed (127.23.186.193:44989): tablet 807385a1ad2947b8b64048aeced6640a (table test-table [id=77456f2d19a643fe8be6f6961a0eb873]) successfully deleted
W20260811 14:33:57.521533 10827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:57.521763 10827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:57.521790 10827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:57.522943 10827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:57.522984 10827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:57.523010 10827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:33:57.524489 10827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:42857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=35649
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:57.524775 10827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:57.525009 10827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:57.530572 10838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.530573 10836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.530573 10840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:57.530838 10827 server_base.cc:1034] running on GCE node
I20260811 14:33:57.531260 10827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:57.531574 10827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:57.532806 10827 hybrid_clock.cc:648] HybridClock initialized: now 1786458837532786 us; error 29 us; skew 500 ppm
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:57.535686 10827 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:33:57.536249 10827 webserver.cc:460] Webserver started at http://127.23.186.194:35649/ using document root <none> and password file <none>
I20260811 14:33:57.536523 10827 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:57.536634 10827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:57.538250 10827 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:57.539032 10848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.539279 10827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:57.539397 10827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "7c809573d8a6460498758e2ecaad1075"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "c1d430a566923d77f95853cae7c4d7d8"
server_key_iv: "db0b173a45a509facd1d4ab3c789b182"
server_key_version: "encryptionkey@0"
I20260811 14:33:57.539790 10827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:57.551366 10827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:57.553818 10827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:57.553998 10827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:57.554262 10827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:57.554610 10827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:57.554677 10827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.554723 10827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:57.554754 10827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.561149 10827 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:42857
I20260811 14:33:57.561192 10976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:42857 every 8 connection(s)
I20260811 14:33:57.561573 10827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:57.570274 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10827
I20260811 14:33:57.570591 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:45617
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:38929
--webserver_port=39885
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:33:57.571862 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.562469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:52071 (local address 127.23.186.254:38929)
0811 14:33:57.562544 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.562547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.562736 (+ 189us) server_negotiation.cc:407] Connection header received
0811 14:33:57.563192 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.563193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.563208 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.563280 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:57.564280 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.564820 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.565901 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.566149 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.566212 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.568477 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:57.568478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:57.568478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:57.568487 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:57.570014 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.570634 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.570635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.570636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.570740 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.571038 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.571039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.571040 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.571217 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:57.571298 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.571605 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.571719 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:33:57.571906 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.562374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:52071)
0811 14:33:57.562553 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:57.562610 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:57.563094 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:57.563334 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:57.563337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:57.563957 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:57.564164 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.564169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.564934 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.564935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.565805 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.565808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.566247 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.566248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.566365 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.566908 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:57.566916 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:57.568374 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:57.570125 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.570133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.570135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.570519 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.570832 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.570833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.570834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.570960 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.571298 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:57.571305 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:57.571459 (+ 154us) client_negotiation.cc:769] Sending connection context
0811 14:33:57.571570 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 14:33:57.571717 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260811 14:33:57.572535 10977 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38929
I20260811 14:33:57.572680 10977 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:57.572974 10977 heartbeater.cc:511] Master 127.23.186.254:38929 requested a full tablet report, sending...
I20260811 14:33:57.573483 10609 ts_manager.cc:194] Registered new tserver with Master: 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857)
I20260811 14:33:57.573899 10609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:52071
W20260811 14:33:57.659969 10980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:33:57.660214 10980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:33:57.660241 10980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:33:57.661518 10980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:33:57.661567 10980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:33:57.661597 10980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:33:57.663136 10980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:45617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=39885
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:33:57.663422 10980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:33:57.663676 10980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:33:57.670171 10989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.670166 10987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:33:57.670190 10986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:33:57.670562 10980 server_base.cc:1034] running on GCE node
I20260811 14:33:57.670804 10980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:33:57.671072 10980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:33:57.672254 10980 hybrid_clock.cc:648] HybridClock initialized: now 1786458837672220 us; error 37 us; skew 500 ppm
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:33:57.675189 10980 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:33:57.675644 10980 webserver.cc:460] Webserver started at http://127.23.186.195:39885/ using document root <none> and password file <none>
I20260811 14:33:57.675872 10980 fs_manager.cc:359] Metadata directory not provided
I20260811 14:33:57.675930 10980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:33:57.677413 10980 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:57.678139 10997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.678346 10980 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:33:57.678439 10980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "58a1f1fb5fe944d28f402fe090706160"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "a1d04c39ef16eef3d9265ad8dfed715a"
server_key_iv: "a58a998ef0d1529319ec692face06c42"
server_key_version: "encryptionkey@0"
I20260811 14:33:57.678800 10980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:33:57.728242 10980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:33:57.730792 10980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:33:57.730945 10980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:33:57.731176 10980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:33:57.731566 10980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:33:57.731613 10980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.731643 10980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:33:57.731701 10980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:33:57.738927 10980 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:45617
I20260811 14:33:57.739012 11126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:45617 every 8 connection(s)
I20260811 14:33:57.739416 10980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
Aug 11 14:33:57 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458837, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:33:57.749013 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 10980
I20260811 14:33:57.750093 10825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.740348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:54999 (local address 127.23.186.254:38929)
0811 14:33:57.740455 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.740457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.740643 (+ 186us) server_negotiation.cc:407] Connection header received
0811 14:33:57.741233 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.741234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.741247 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.741310 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:57.742379 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.742918 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.744019 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.744229 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.744316 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.746842 (+ 2526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:57.746844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:57.746844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:57.746853 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:57.748244 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.748784 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.748785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.748786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.748843 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.749212 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.749213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.749214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.749427 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:57.749532 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.749864 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.749974 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:33:57.750339 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.740306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:54999)
0811 14:33:57.740508 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:57.740548 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:57.741109 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:57.741390 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:57.741394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:57.742041 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:33:57.742257 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.742263 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.743041 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.743042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.743905 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.743908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.744326 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.744327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.744480 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.745093 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:33:57.745103 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:33:57.746711 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:33:57.748332 (+ 1621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.748341 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.748344 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.748626 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.748917 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:33:57.748918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:33:57.748918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:33:57.749095 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:33:57.749539 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:33:57.749548 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:33:57.749720 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 14:33:57.749818 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 14:33:57.750058 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":94,"threads_started":1}
I20260811 14:33:57.751179 11127 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38929
I20260811 14:33:57.751408 11127 heartbeater.cc:464] Registering TS with master...
I20260811 14:33:57.751667 11127 heartbeater.cc:511] Master 127.23.186.254:38929 requested a full tablet report, sending...
I20260811 14:33:57.752182 10609 ts_manager.cc:194] Registered new tserver with Master: 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
I20260811 14:33:57.752720 10609 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:54999
I20260811 14:33:57.757508 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.749527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51334 (local address 127.23.186.254:38929)
0811 14:33:57.749686 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.749689 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.749855 (+ 166us) server_negotiation.cc:407] Connection header received
0811 14:33:57.749915 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.749916 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.749929 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.750022 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:57.750832 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.751661 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.752880 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.753195 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.753274 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.754635 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:57.754637 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:57.754637 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:57.754645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:57.756074 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.756541 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.756543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.756543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.756631 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.756951 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.756952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.756953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.757138 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:57.757275 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.757345 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.757431 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":69,"threads_started":1}
I20260811 14:33:57.758459 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:33:57.761932 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.758962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.23.186.254:38929)
0811 14:33:57.759051 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.759054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.759167 (+ 113us) server_negotiation.cc:407] Connection header received
0811 14:33:57.759252 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.759253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.759294 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.759386 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:57.759553 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.760058 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.761089 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.761321 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.761403 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.761556 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:57.761623 (+ 67us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 14:33:57.761852 (+ 229us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51338: token expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 14:33:57.762027 11130 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51338: token expired
W20260811 14:33:57.762151 10525 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.23.186.254:38929)
I20260811 14:33:57.768934 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.762449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51344 (local address 127.23.186.254:38929)
0811 14:33:57.762506 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.762508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.762517 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:57.762582 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.762583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.762592 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.762646 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:33:57.763169 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.763645 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.764402 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.764616 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.764694 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.766067 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:33:57.766069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:33:57.766069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:33:57.766078 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:33:57.767626 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.768045 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.768047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.768047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.768109 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:33:57.768327 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:33:57.768328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:33:57.768329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:33:57.768484 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:33:57.768603 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.768733 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.768828 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:33:57.770336 10609 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:33:57.771958 10609 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:33:57.774507 10745 tablet_service.cc:1467] Processing CreateTablet for tablet 71d3c66a037f4d38bcfdb0bd113c160a (DEFAULT_TABLE table=test-table [id=c3b7b6a2c29a4d408b93083d4a4ffb72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:57.774742 10745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71d3c66a037f4d38bcfdb0bd113c160a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:57.776343 10698 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Bootstrap starting.
I20260811 14:33:57.776782 10698 tablet_bootstrap.cc:654] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:57.777460 10698 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: No bootstrap required, opened a new log
I20260811 14:33:57.777506 10698 ts_tablet_manager.cc:1397] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260811 14:33:57.777886 10698 raft_consensus.cc:348] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:57.777956 10698 raft_consensus.cc:374] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:57.777987 10698 raft_consensus.cc:729] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a233dde4dd408281418532370591ed, State: Initialized, Role: FOLLOWER
I20260811 14:33:57.778064 10698 consensus_queue.cc:260] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:57.778164 10698 ts_tablet_manager.cc:1428] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 14:33:57.778585 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.774187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:45617 (local address 127.0.0.1:50820)
0811 14:33:57.774231 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:57.774244 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:57.774284 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:57.774806 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:57.774807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:57.774809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:57.774868 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.774870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.775979 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.775980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.777307 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.777310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.778293 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.778294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.778369 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.778397 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 14:33:57.778460 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 14:33:57.778507 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 14:33:57.779006 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.774360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.23.186.195:45617)
0811 14:33:57.774597 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.774602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.774613 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:33:57.774659 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.774662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.774680 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.774793 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:57.774962 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.775870 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.777415 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.778218 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.778299 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.778368 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.778867 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.778925 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":131,"threads_started":1}
I20260811 14:33:57.779835 11049 tablet_service.cc:1467] Processing CreateTablet for tablet 71d3c66a037f4d38bcfdb0bd113c160a (DEFAULT_TABLE table=test-table [id=c3b7b6a2c29a4d408b93083d4a4ffb72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:57.780107 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.774188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:42857 (local address 127.0.0.1:37064)
0811 14:33:57.776345 (+ 2157us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:57.776358 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:57.776439 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:57.776680 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:57.776681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:57.776684 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:57.776758 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.776760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.778043 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.778044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.779099 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:57.779102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.779860 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.779861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:57.779925 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.779930 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:33:57.779990 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:33:57.780034 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2141,"mutex_wait_us":362,"thread_start_us":61,"threads_started":1}
I20260811 14:33:57.780128 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:57.774351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37064 (local address 127.23.186.194:42857)
0811 14:33:57.774963 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:57.774967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:57.776506 (+ 1539us) server_negotiation.cc:407] Connection header received
0811 14:33:57.776561 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:57.776563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:57.776581 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:57.776660 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:57.776839 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.777950 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.779181 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:57.779771 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:57.779862 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:57.779926 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:57.780006 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:33:57.780051 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":155,"threads_started":1}
I20260811 14:33:57.780263 11049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71d3c66a037f4d38bcfdb0bd113c160a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:57.781023 10901 tablet_service.cc:1467] Processing CreateTablet for tablet 71d3c66a037f4d38bcfdb0bd113c160a (DEFAULT_TABLE table=test-table [id=c3b7b6a2c29a4d408b93083d4a4ffb72]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:33:57.781426 10901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71d3c66a037f4d38bcfdb0bd113c160a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:33:57.784343 11138 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Bootstrap starting.
I20260811 14:33:57.785107 11138 tablet_bootstrap.cc:654] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:57.785208 11139 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Bootstrap starting.
I20260811 14:33:57.785734 11138 log.cc:824] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:57.785811 11139 tablet_bootstrap.cc:654] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Neither blocks nor log segments found. Creating new log.
I20260811 14:33:57.786451 11139 log.cc:824] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Log is configured to *not* fsync() on all Append() calls
I20260811 14:33:57.786688 11138 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: No bootstrap required, opened a new log
I20260811 14:33:57.786906 11138 ts_tablet_manager.cc:1397] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:33:57.787499 11139 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: No bootstrap required, opened a new log
I20260811 14:33:57.787592 11139 ts_tablet_manager.cc:1397] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:33:57.788678 11138 raft_consensus.cc:348] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:57.788934 11138 raft_consensus.cc:374] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:57.788960 11138 raft_consensus.cc:729] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58a1f1fb5fe944d28f402fe090706160, State: Initialized, Role: FOLLOWER
I20260811 14:33:57.789111 11138 consensus_queue.cc:260] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:57.789216 11139 raft_consensus.cc:348] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:57.789412 11139 raft_consensus.cc:374] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:33:57.789444 11138 ts_tablet_manager.cc:1428] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 14:33:57.789449 11139 raft_consensus.cc:729] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c809573d8a6460498758e2ecaad1075, State: Initialized, Role: FOLLOWER
I20260811 14:33:57.789465 11127 heartbeater.cc:503] Master 127.23.186.254:38929 was elected leader, sending a full tablet report...
I20260811 14:33:57.789565 11139 consensus_queue.cc:260] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:57.789853 11139 ts_tablet_manager.cc:1428] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:33:57.789897 10977 heartbeater.cc:503] Master 127.23.186.254:38929 was elected leader, sending a full tablet report...
W20260811 14:33:57.812345 10978 tablet.cc:2369] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:57.922163 10820 tablet.cc:2369] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:33:57.990239 11128 tablet.cc:2369] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:33:58.015316 10828 raft_consensus.cc:482] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:33:58.015569 10828 raft_consensus.cc:504] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:58.015978 10828 leader_election.cc:288] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
I20260811 14:33:58.020870 11135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:58.016500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:39659 (local address 127.23.186.195:45617)
0811 14:33:58.016556 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:58.016559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:58.016641 (+ 82us) server_negotiation.cc:407] Connection header received
0811 14:33:58.016738 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:58.016739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:58.016751 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:58.016833 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:58.017033 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.017831 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.019750 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.020366 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.020462 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:58.020472 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:58.020738 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 14:33:58.020787 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:33:58.020928 11145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:58.016369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:45617 (local address 127.23.186.193:39659)
0811 14:33:58.016565 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:58.016576 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:58.016637 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:58.016857 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:58.016858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:58.016861 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:58.016939 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:58.016942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.017990 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.017992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:58.019604 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:58.019609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.020504 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.020506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:58.020596 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:58.020633 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 14:33:58.020732 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 14:33:58.020807 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":109,"spinlock_wait_cycles":18176,"thread_start_us":101,"threads_started":1}
I20260811 14:33:58.021026 11134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:58.016506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:43709 (local address 127.23.186.194:42857)
0811 14:33:58.016556 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:58.016559 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:58.016640 (+ 81us) server_negotiation.cc:407] Connection header received
0811 14:33:58.016741 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:58.016742 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:58.016751 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:58.016833 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:33:58.017033 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.017831 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.019750 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.020385 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.020462 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:58.020472 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:58.020890 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 14:33:58.020946 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:33:58.020934 11144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:58.016368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:42857 (local address 127.23.186.193:43709)
0811 14:33:58.016536 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:58.016549 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 14:33:58.016637 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:33:58.016857 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:33:58.016858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:33:58.016862 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:33:58.016945 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:58.016946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.017990 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.017992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:58.019604 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:33:58.019609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.020506 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.020507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:33:58.020593 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:58.020631 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 14:33:58.020732 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 14:33:58.020807 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":155,"thread_start_us":85,"threads_started":1}
I20260811 14:33:58.021333 11073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d3c66a037f4d38bcfdb0bd113c160a" candidate_uuid: "b0a233dde4dd408281418532370591ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a1f1fb5fe944d28f402fe090706160" is_pre_election: true
I20260811 14:33:58.021399 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d3c66a037f4d38bcfdb0bd113c160a" candidate_uuid: "b0a233dde4dd408281418532370591ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c809573d8a6460498758e2ecaad1075" is_pre_election: true
I20260811 14:33:58.021533 10922 raft_consensus.cc:2454] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a233dde4dd408281418532370591ed in term 0.
I20260811 14:33:58.021533 11073 raft_consensus.cc:2454] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b0a233dde4dd408281418532370591ed in term 0.
I20260811 14:33:58.021934 10692 leader_election.cc:302] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a1f1fb5fe944d28f402fe090706160, b0a233dde4dd408281418532370591ed; no voters:
I20260811 14:33:58.022219 10828 raft_consensus.cc:2781] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:33:58.022321 10828 raft_consensus.cc:482] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:33:58.022343 10828 raft_consensus.cc:3037] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:58.023317 10828 raft_consensus.cc:504] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:58.023514 10828 leader_election.cc:288] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 1 election: Requested vote from peers 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
I20260811 14:33:58.023732 11073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d3c66a037f4d38bcfdb0bd113c160a" candidate_uuid: "b0a233dde4dd408281418532370591ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58a1f1fb5fe944d28f402fe090706160"
I20260811 14:33:58.023792 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71d3c66a037f4d38bcfdb0bd113c160a" candidate_uuid: "b0a233dde4dd408281418532370591ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c809573d8a6460498758e2ecaad1075"
I20260811 14:33:58.023828 11073 raft_consensus.cc:3037] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:58.023909 10922 raft_consensus.cc:3037] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:33:58.024652 11073 raft_consensus.cc:2454] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a233dde4dd408281418532370591ed in term 1.
I20260811 14:33:58.024667 10922 raft_consensus.cc:2454] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b0a233dde4dd408281418532370591ed in term 1.
I20260811 14:33:58.024868 10692 leader_election.cc:302] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 58a1f1fb5fe944d28f402fe090706160, b0a233dde4dd408281418532370591ed; no voters:
I20260811 14:33:58.025025 10828 raft_consensus.cc:2781] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:33:58.025197 10828 raft_consensus.cc:686] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 LEADER]: Becoming Leader. State: Replica: b0a233dde4dd408281418532370591ed, State: Running, Role: LEADER
I20260811 14:33:58.025270 10828 consensus_queue.cc:237] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:33:58.026119 10609 catalog_manager.cc:5495] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed reported cstate change: term changed from 0 to 1, leader changed from <none> to b0a233dde4dd408281418532370591ed (127.23.186.193). New cstate: current_term: 1 leader_uuid: "b0a233dde4dd408281418532370591ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } health_report { overall_health: HEALTHY } } }
I20260811 14:33:58.067512 11149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:58.064677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.23.186.193:44989)
0811 14:33:58.064910 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 14:33:58.064912 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:33:58.064921 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:33:58.064977 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:33:58.064977 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:33:58.065000 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:33:58.065105 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:33:58.065246 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.065764 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.066556 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:33:58.066793 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:33:58.066885 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:33:58.067088 (+ 203us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:33:58.067222 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:33:58.067323 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 14:33:58.067380 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 14:33:58.067404 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":154,"threads_started":1}
I20260811 14:33:58.069420 10922 raft_consensus.cc:1261] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Refusing update from remote peer b0a233dde4dd408281418532370591ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:58.069420 11073 raft_consensus.cc:1261] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Refusing update from remote peer b0a233dde4dd408281418532370591ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:33:58.069841 11146 consensus_queue.cc:1035] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:58.070010 10828 consensus_queue.cc:1035] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:33:58.073033 11150 mvcc.cc:205] Tried to move back new op lower bound from 7317335400729468928 to 7317335400552280064. Current Snapshot: MvccSnapshot[applied={T|T < 7317335400729468928}]
I20260811 14:33:58.073709 11151 mvcc.cc:205] Tried to move back new op lower bound from 7317335400729468928 to 7317335400552280064. Current Snapshot: MvccSnapshot[applied={T|T < 7317335400729468928}]
I20260811 14:33:58.073998 11152 mvcc.cc:205] Tried to move back new op lower bound from 7317335400729468928 to 7317335400552280064. Current Snapshot: MvccSnapshot[applied={T|T < 7317335400729468928}]
I20260811 14:33:58.075742 10607 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:58.075852 10607 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:33:58.076802 10607 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Sending DeleteTablet for 3 replicas of tablet 71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:33:58.077173 10745 tablet_service.cc:1514] Processing DeleteTablet for tablet 71d3c66a037f4d38bcfdb0bd113c160a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:58 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:57468
I20260811 14:33:58.077276 11049 tablet_service.cc:1514] Processing DeleteTablet for tablet 71d3c66a037f4d38bcfdb0bd113c160a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:58 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:50820
I20260811 14:33:58.077450 11160 tablet_replica.cc:330] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: stopping tablet replica
I20260811 14:33:58.077509 11160 raft_consensus.cc:2229] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:33:58.077545 10901 tablet_service.cc:1514] Processing DeleteTablet for tablet 71d3c66a037f4d38bcfdb0bd113c160a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:33:58 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:37064
I20260811 14:33:58.077605 11160 raft_consensus.cc:2258] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:58.077791 11162 tablet_replica.cc:330] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: stopping tablet replica
I20260811 14:33:58.077883 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10649
I20260811 14:33:58.077950 11160 ts_tablet_manager.cc:1905] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:58.078059 11162 raft_consensus.cc:2229] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:58.078241 11162 raft_consensus.cc:2258] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:58.081650 11162 ts_tablet_manager.cc:1905] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:58.084133 11162 ts_tablet_manager.cc:1918] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:58.084347 11162 log.cc:1196] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:33:58.084816 11162 ts_tablet_manager.cc:1939] T 71d3c66a037f4d38bcfdb0bd113c160a P 7c809573d8a6460498758e2ecaad1075: Deleting consensus metadata
I20260811 14:33:58.085429 11161 tablet_replica.cc:330] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: stopping tablet replica
I20260811 14:33:58.085533 10589 catalog_manager.cc:4841] TS 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857): tablet 71d3c66a037f4d38bcfdb0bd113c160a (table test-table [id=c3b7b6a2c29a4d408b93083d4a4ffb72]) successfully deleted
I20260811 14:33:58.085621 11161 raft_consensus.cc:2229] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:33:58.085800 11161 raft_consensus.cc:2258] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:33:58.086531 11161 ts_tablet_manager.cc:1905] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:33:58.088872 11161 ts_tablet_manager.cc:1918] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:33:58.089056 11161 log.cc:1196] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:33:58.089453 11161 ts_tablet_manager.cc:1939] T 71d3c66a037f4d38bcfdb0bd113c160a P 58a1f1fb5fe944d28f402fe090706160: Deleting consensus metadata
I20260811 14:33:58.090103 10591 catalog_manager.cc:4841] TS 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617): tablet 71d3c66a037f4d38bcfdb0bd113c160a (table test-table [id=c3b7b6a2c29a4d408b93083d4a4ffb72]) successfully deleted
W20260811 14:33:58.091900 10594 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:44989 (error 108)
I20260811 14:33:58.092392 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10827
I20260811 14:33:58.092535 11136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:33:58.092387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44989 (local address 127.0.0.1:57484)
0811 14:33:58.092443 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0811 14:33:58.092490 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:44989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:33:58.093575 10594 catalog_manager.cc:4584] TS b0a233dde4dd408281418532370591ed (127.23.186.193:44989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71d3c66a037f4d38bcfdb0bd113c160a: Network error: Client connection negotiation failed: client connection to 127.23.186.193:44989: connect: Connection refused (error 111)
I20260811 14:33:58.099030 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10980
I20260811 14:33:58.111039 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 10569
I20260811 14:34:18.119621 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38929
--webserver_interface=127.23.186.254
--webserver_port=44263
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:38929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:18.204236 11166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:18.204471 11166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:18.204495 11166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:18.205967 11166 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 14:34:18.206032 11166 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 14:34:18.206046 11166 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 14:34:18.206059 11166 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:18.206072 11166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:18.206086 11166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:18.206099 11166 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:18.206111 11166 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:18.207768 11166 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:38929
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=44263
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:18.207988 11166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:18.208181 11166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:18.213253 11171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.213263 11172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.213263 11174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:18.213737 11166 server_base.cc:1034] running on GCE node
I20260811 14:34:18.213951 11166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:18.214213 11166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:18.215368 11166 hybrid_clock.cc:648] HybridClock initialized: now 1786458858215357 us; error 38 us; skew 500 ppm
Aug 11 14:34:18 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458858, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:18.218072 11166 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:34:18.218556 11166 webserver.cc:460] Webserver started at http://127.23.186.254:44263/ using document root <none> and password file <none>
I20260811 14:34:18.218757 11166 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:18.218801 11166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:18.220288 11166 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:18.221029 11181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.221215 11166 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:18.221316 11166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "03ed8ac4bea64e79936331529deee5e5"
format_stamp: "Formatted at 2026-08-11 14:33:35 on dist-test-slave-cbsf"
server_key: "80d8779a0866c5cfa3dc0fa257a84641"
server_key_iv: "554dbe8b76a2bdb66864ab8040628fe0"
server_key_version: "encryptionkey@0"
I20260811 14:34:18.221692 11166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:18.261051 11166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:18.263527 11166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:18.263653 11166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:18.267741 11166 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:38929
I20260811 14:34:18.267827 11244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:38929 every 8 connection(s)
I20260811 14:34:18.268220 11166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:18.269863 11245 sys_catalog.cc:263] Verifying existing consensus state
I20260811 14:34:18.270556 11245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap starting.
I20260811 14:34:18.276786 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 11166
I20260811 14:34:18.277146 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:44989
--local_ip_for_outbound_sockets=127.23.186.193
--tserver_master_addrs=127.23.186.254:38929
--webserver_port=45259
--webserver_interface=127.23.186.193
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:34:18.279230 11245 log.cc:824] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:18.281774 11245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:34:18.282109 11245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Bootstrap complete.
I20260811 14:34:18.284425 11245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:34:18.284690 11245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ed8ac4bea64e79936331529deee5e5, State: Initialized, Role: FOLLOWER
I20260811 14:34:18.284926 11245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:34:18.285033 11245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:18.285087 11245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:18.285135 11245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 2 FOLLOWER]: Advancing to term 3
I20260811 14:34:18.286175 11245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:34:18.286299 11245 leader_election.cc:302] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ed8ac4bea64e79936331529deee5e5; no voters:
I20260811 14:34:18.286512 11245 leader_election.cc:288] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 14:34:18.286556 11249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 14:34:18.286680 11249 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [term 3 LEADER]: Becoming Leader. State: Replica: 03ed8ac4bea64e79936331529deee5e5, State: Running, Role: LEADER
I20260811 14:34:18.286782 11249 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } }
I20260811 14:34:18.286809 11245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:18.287082 11249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ed8ac4bea64e79936331529deee5e5. Latest consensus state: current_term: 3 leader_uuid: "03ed8ac4bea64e79936331529deee5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } } }
I20260811 14:34:18.287153 11249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:18.287492 11250 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "03ed8ac4bea64e79936331529deee5e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ed8ac4bea64e79936331529deee5e5" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38929 } } }
I20260811 14:34:18.287598 11250 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:18.287811 11253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:18.288820 11253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:18.288933 11253 catalog_manager.cc:1259] Loaded cluster ID: d00b52b83e1240089cc5c6853d69a92d
I20260811 14:34:18.289201 11253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:18.290491 11253 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:18.290676 11253 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Loaded TSK: 0
I20260811 14:34:18.290721 11253 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 03ed8ac4bea64e79936331529deee5e5: Loaded TSK: 1
F20260811 14:34:18.290946 11253 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f3df77f09ad google::LogMessage::Fail() at ??:0
@ 0x7f3df77f388a google::LogMessage::SendToLog() at ??:0
@ 0x7f3df77f04de google::LogMessage::Flush() at ??:0
@ 0x7f3df77f1a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f3df9f5d104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f3df9f5ad41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f3dff74a1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f3dff74a48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f3dff74a594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f3dff710c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f3dff71aad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f3df96be0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f3df96b3954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f3dfa5176db start_thread at ??:0
@ 0x7f3df62ee61f clone at ??:0
W20260811 14:34:18.382611 11247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:18.382858 11247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:18.382895 11247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:18.384066 11247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:18.384137 11247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:18.384182 11247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:18.385751 11247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:44989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=45259
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:18.386052 11247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:18.386296 11247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:18.391637 11294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:18.391785 11247 server_base.cc:1034] running on GCE node
W20260811 14:34:18.391654 11292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.391896 11291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:18.392117 11247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:18.392378 11247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:18.393571 11247 hybrid_clock.cc:648] HybridClock initialized: now 1786458858393558 us; error 32 us; skew 500 ppm
Aug 11 14:34:18 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458858, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:18.396220 11247 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:34:18.396775 11247 webserver.cc:460] Webserver started at http://127.23.186.193:45259/ using document root <none> and password file <none>
I20260811 14:34:18.397049 11247 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:18.397099 11247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:18.398488 11247 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:18.399278 11301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.399473 11247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:18.399567 11247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "b0a233dde4dd408281418532370591ed"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "e01f5ffb258f35f33f7411e0783e8924"
server_key_iv: "531abfaa13aaa72a1c343cc43d51bd23"
server_key_version: "encryptionkey@0"
I20260811 14:34:18.399928 11247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:18.430773 11247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:18.433113 11247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:18.433275 11247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:18.433549 11247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:18.434151 11310 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 14:34:18.436251 11247 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 14:34:18.436321 11247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260811 14:34:18.436365 11247 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 14:34:18.437047 11247 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 14:34:18.437104 11247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260811 14:34:18.437184 11310 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Bootstrap starting.
I20260811 14:34:18.439113 11310 log.cc:824] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:18.442255 11310 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 14:34:18.442579 11310 tablet_bootstrap.cc:492] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Bootstrap complete.
I20260811 14:34:18.442752 11310 ts_tablet_manager.cc:1397] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 14:34:18.444579 11247 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:44989
I20260811 14:34:18.444620 11433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:44989 every 8 connection(s)
I20260811 14:34:18.444988 11247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:18.444840 11310 raft_consensus.cc:348] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:34:18.445184 11310 raft_consensus.cc:729] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0a233dde4dd408281418532370591ed, State: Initialized, Role: FOLLOWER
I20260811 14:34:18.445498 11310 consensus_queue.cc:260] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:34:18.445796 11310 ts_tablet_manager.cc:1428] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:18.454260 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 11247
I20260811 14:34:18.454607 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:42857
--local_ip_for_outbound_sockets=127.23.186.194
--tserver_master_addrs=127.23.186.254:38929
--webserver_port=35649
--webserver_interface=127.23.186.194
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:18.502837 11437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:34:18.445791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:38167)
0811 14:34:18.446006 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.446043 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:18.446156 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:18.502600 (+ 56444us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":74,"threads_started":1}
W20260811 14:34:18.503091 11434 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:38929 (0 consecutive failures): Network error: Failed to ping master at 127.23.186.254:38929: Client connection negotiation failed: client connection to 127.23.186.254:38929: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 14:34:18.503472 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.503349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:47951)
0811 14:34:18.503408 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.503452 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:34:18.503880 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.503788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:44701)
0811 14:34:18.503834 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.503859 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:34:18.504000 11434 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 14:34:18.541188 11440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:18.541436 11440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:18.541481 11440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:18.542670 11440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:18.542737 11440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:18.542784 11440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:18.544337 11440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:42857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=35649
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:18.544637 11440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:18.544898 11440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:18.550282 11450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.550268 11448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.550247 11447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:18.550643 11440 server_base.cc:1034] running on GCE node
I20260811 14:34:18.550904 11440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:18.551203 11440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:18.552376 11440 hybrid_clock.cc:648] HybridClock initialized: now 1786458858552362 us; error 33 us; skew 500 ppm
Aug 11 14:34:18 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458858, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:18.555259 11440 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:34:18.555821 11440 webserver.cc:460] Webserver started at http://127.23.186.194:35649/ using document root <none> and password file <none>
I20260811 14:34:18.556120 11440 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:18.556198 11440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:18.557830 11440 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:18.558516 11459 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.558699 11440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:18.558818 11440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "7c809573d8a6460498758e2ecaad1075"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "c1d430a566923d77f95853cae7c4d7d8"
server_key_iv: "db0b173a45a509facd1d4ab3c789b182"
server_key_version: "encryptionkey@0"
I20260811 14:34:18.559224 11440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:18.592834 11440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:18.595171 11440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:18.595378 11440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:18.595654 11440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:18.596010 11440 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:18.596072 11440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.596117 11440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:18.596148 11440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.602559 11440 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:42857
I20260811 14:34:18.602610 11583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:42857 every 8 connection(s)
I20260811 14:34:18.603098 11440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:34:18.604295 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.603818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:52287)
0811 14:34:18.604067 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.604154 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":120,"threads_started":1}
W20260811 14:34:18.604746 11585 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:38929 (0 consecutive failures): Network error: Failed to ping master at 127.23.186.254:38929: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
I20260811 14:34:18.605058 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.604955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:33447)
0811 14:34:18.605004 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.605029 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 14:34:18.605337 11587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.605251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:37217)
0811 14:34:18.605295 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.605317 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260811 14:34:18.605458 11585 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:34:18.611701 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 11440
I20260811 14:34:18.611975 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:45617
--local_ip_for_outbound_sockets=127.23.186.195
--tserver_master_addrs=127.23.186.254:38929
--webserver_port=39885
--webserver_interface=127.23.186.195
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:18.696650 11588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:18.696976 11588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:18.697021 11588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:18.698585 11588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:18.698639 11588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:18.698686 11588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:18.700243 11588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:45617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=39885
--tserver_master_addrs=127.23.186.254:38929
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:18.700531 11588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:18.700848 11588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:18.706418 11597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.706424 11595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:18.706447 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:18.706441 11588 server_base.cc:1034] running on GCE node
I20260811 14:34:18.706938 11588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:18.707194 11588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:18.708344 11588 hybrid_clock.cc:648] HybridClock initialized: now 1786458858708332 us; error 30 us; skew 500 ppm
Aug 11 14:34:18 dist-test-slave-cbsf krb5kdc[9917](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458858, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:18.711004 11588 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:34:18.711472 11588 webserver.cc:460] Webserver started at http://127.23.186.195:39885/ using document root <none> and password file <none>
I20260811 14:34:18.711714 11588 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:18.711763 11588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:18.713248 11588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:34:18.713837 11605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.714021 11588 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:18.714115 11588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "58a1f1fb5fe944d28f402fe090706160"
format_stamp: "Formatted at 2026-08-11 14:33:36 on dist-test-slave-cbsf"
server_key: "a1d04c39ef16eef3d9265ad8dfed715a"
server_key_iv: "a58a998ef0d1529319ec692face06c42"
server_key_version: "encryptionkey@0"
I20260811 14:34:18.714476 11588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:18.724421 11588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:18.726810 11588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:18.727003 11588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:18.727241 11588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:18.727589 11588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:18.727644 11588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.727687 11588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:18.727715 11588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:18.733588 11588 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:45617
I20260811 14:34:18.733668 11728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:45617 every 8 connection(s)
I20260811 14:34:18.734050 11588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:18.735181 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.734790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:44521)
0811 14:34:18.734958 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.735035 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
W20260811 14:34:18.735641 11729 heartbeater.cc:644] Failed to heartbeat to 127.23.186.254:38929 (0 consecutive failures): Network error: Failed to ping master at 127.23.186.254:38929: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
I20260811 14:34:18.735932 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.735839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:42663)
0811 14:34:18.735886 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.735914 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 14:34:18.736227 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:18.736151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:37709)
0811 14:34:18.736191 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:18.736212 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260811 14:34:18.736352 11729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 14:34:18.738315 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 11588
I20260811 14:34:19.504609 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.504372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:40065)
0811 14:34:19.504500 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:19.504541 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260811 14:34:19.606015 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.605788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:39779)
0811 14:34:19.605915 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:19.605958 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":79,"threads_started":1}
I20260811 14:34:19.671917 11736 raft_consensus.cc:482] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:19.672067 11736 raft_consensus.cc:504] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:34:19.672510 11736 leader_election.cc:288] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
Aug 11 14:34:19 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458858, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.194@KRBTEST.COM
Aug 11 14:34:19 dist-test-slave-cbsf krb5kdc[9917](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458858, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.195@KRBTEST.COM
I20260811 14:34:19.683744 11740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.673011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:39413 (local address 127.23.186.195:45617)
0811 14:34:19.673186 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:19.673203 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:19.673220 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:34:19.673621 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:19.673624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:19.673673 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:19.673825 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:19.674684 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.675581 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.676519 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.676996 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.677133 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:19.680234 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:19.680237 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:19.680237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:19.680247 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:19.681998 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682412 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:19.682416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:19.682416 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:19.682526 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682815 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:19.682816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:19.682816 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:19.683082 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:19.683171 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:19.683480 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 14:34:19.683620 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":102,"threads_started":1}
I20260811 14:34:19.683748 11738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.672982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:40159 (local address 127.23.186.194:42857)
0811 14:34:19.673105 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:19.673117 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:19.673150 (+ 33us) server_negotiation.cc:407] Connection header received
0811 14:34:19.673609 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:19.673612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:19.673654 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:19.673818 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:19.674570 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.675459 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.676519 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.676998 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.677144 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:19.679742 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:19.679747 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:19.679747 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:19.679759 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:19.681885 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682445 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:19.682448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:19.682448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:19.682538 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682812 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:19.682812 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:19.682813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:19.683075 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:19.683171 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:19.683491 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 14:34:19.683623 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":63,"threads_started":1}
I20260811 14:34:19.683743 11737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.672773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:42857 (local address 127.23.186.193:40159)
0811 14:34:19.672905 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:19.672917 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:19.672989 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:19.673856 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:19.673859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:19.674234 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:19.674435 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:19.674448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.675572 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.675573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:19.676399 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:19.676402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.677093 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.677094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:19.677250 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:19.677918 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:19.677923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:19.679557 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:19.681993 (+ 2436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682005 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:19.682008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:19.682337 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:19.682634 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682635 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:19.682636 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:19.682740 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:19.683162 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:19.683171 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:19.683298 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 14:34:19.683411 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 14:34:19.683632 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":23,"thread_start_us":59,"threads_started":1}
I20260811 14:34:19.683871 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.672773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:45617 (local address 127.23.186.193:39413)
0811 14:34:19.672840 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:19.672855 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:19.672953 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:19.673856 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:19.673859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:19.674488 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:19.674551 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:19.674553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.675690 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.675691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:19.676401 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:19.676403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:19.677102 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:19.677103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:19.677249 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:19.677864 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:19.677873 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:19.679934 (+ 2061us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:19.682098 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:19.682099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:19.682326 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:19.682617 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:19.682618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:19.682618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:19.682738 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:19.683164 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:19.683171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:19.683298 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 14:34:19.683410 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 14:34:19.683722 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":150}
W20260811 14:34:19.684965 11302 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
W20260811 14:34:19.685092 11303 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:34:19.685144 11303 leader_election.cc:302] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a233dde4dd408281418532370591ed; no voters: 58a1f1fb5fe944d28f402fe090706160, 7c809573d8a6460498758e2ecaad1075
I20260811 14:34:19.685254 11736 raft_consensus.cc:2726] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:34:19.736948 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:19.736680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:54255)
0811 14:34:19.736857 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:19.736909 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":113,"threads_started":1}
I20260811 14:34:20.505501 11742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:20.505257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:44855)
0811 14:34:20.505401 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:20.505447 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":68,"threads_started":1}
I20260811 14:34:20.606621 11743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:20.606420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:59839)
0811 14:34:20.606543 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:20.606581 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260811 14:34:20.737742 11744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:20.737511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:48917)
0811 14:34:20.737648 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:20.737693 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260811 14:34:21.269598 11746 raft_consensus.cc:482] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:21.269744 11746 raft_consensus.cc:504] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:34:21.269989 11746 leader_election.cc:288] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
W20260811 14:34:21.270479 11303 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
W20260811 14:34:21.270603 11302 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:34:21.270673 11302 leader_election.cc:302] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a233dde4dd408281418532370591ed; no voters: 58a1f1fb5fe944d28f402fe090706160, 7c809573d8a6460498758e2ecaad1075
I20260811 14:34:21.270761 11746 raft_consensus.cc:2726] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:34:21.506280 11747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:21.505999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:44953)
0811 14:34:21.506146 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:21.506214 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":74,"threads_started":1}
I20260811 14:34:21.607419 11748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:21.607187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:57809)
0811 14:34:21.607330 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:21.607374 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":82,"threads_started":1}
I20260811 14:34:21.738613 11749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:21.738338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:34125)
0811 14:34:21.738514 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:21.738563 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260811 14:34:22.507092 11750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:22.506798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:54367)
0811 14:34:22.506945 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:22.507009 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260811 14:34:22.608194 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:22.607956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:58857)
0811 14:34:22.608108 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:22.608149 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260811 14:34:22.739383 11752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:22.739198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:33759)
0811 14:34:22.739314 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:22.739344 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260811 14:34:22.878356 11753 raft_consensus.cc:482] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:22.878463 11753 raft_consensus.cc:504] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:34:22.878715 11753 leader_election.cc:288] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
W20260811 14:34:22.879192 11303 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
W20260811 14:34:22.879278 11302 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:34:22.879344 11302 leader_election.cc:302] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a233dde4dd408281418532370591ed; no voters: 58a1f1fb5fe944d28f402fe090706160, 7c809573d8a6460498758e2ecaad1075
I20260811 14:34:22.879441 11753 raft_consensus.cc:2726] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:34:23.508040 11754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:23.507760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:57633)
0811 14:34:23.507938 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:23.507990 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":71,"threads_started":1}
I20260811 14:34:23.609225 11755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:23.608913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:33181)
0811 14:34:23.609081 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:23.609130 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260811 14:34:23.740061 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:23.739839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:59527)
0811 14:34:23.739972 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:23.740015 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
I20260811 14:34:24.508893 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:24.508602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:45173)
0811 14:34:24.508799 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:24.508844 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":67,"threads_started":1}
I20260811 14:34:24.609941 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:24.609721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:55561)
0811 14:34:24.609860 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:24.609896 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
I20260811 14:34:24.740808 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:24.740536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:36603)
0811 14:34:24.740673 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:24.740758 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260811 14:34:25.509729 11760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:25.509471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:55629)
0811 14:34:25.509632 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:25.509678 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":68,"threads_started":1}
I20260811 14:34:25.610672 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:25.610444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:33163)
0811 14:34:25.610586 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:25.610625 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":66,"threads_started":1}
I20260811 14:34:25.741499 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:25.741270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:33353)
0811 14:34:25.741406 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:25.741445 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260811 14:34:26.510736 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:26.510476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:49185)
0811 14:34:26.510632 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:26.510684 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260811 14:34:26.611384 11764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:26.611159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:43521)
0811 14:34:26.611297 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:26.611338 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":68,"threads_started":1}
I20260811 14:34:26.742252 11765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:26.742018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:57831)
0811 14:34:26.742161 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:26.742207 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":70,"threads_started":1}
I20260811 14:34:27.511643 11766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:27.511361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:33461)
0811 14:34:27.511532 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:27.511583 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":74,"threads_started":1}
I20260811 14:34:27.612098 11767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:27.611863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:42041)
0811 14:34:27.612007 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:27.612049 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260811 14:34:27.669495 11768 raft_consensus.cc:482] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:27.669631 11768 raft_consensus.cc:504] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c809573d8a6460498758e2ecaad1075" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 42857 } } peers { permanent_uuid: "58a1f1fb5fe944d28f402fe090706160" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 45617 } } peers { permanent_uuid: "b0a233dde4dd408281418532370591ed" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44989 } }
I20260811 14:34:27.669831 11768 leader_election.cc:288] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857), 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617)
W20260811 14:34:27.670248 11303 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 58a1f1fb5fe944d28f402fe090706160 (127.23.186.195:45617): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
W20260811 14:34:27.670339 11302 leader_election.cc:341] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 7c809573d8a6460498758e2ecaad1075 (127.23.186.194:42857): Not found: Tablet not found: 71d3c66a037f4d38bcfdb0bd113c160a
I20260811 14:34:27.670405 11302 leader_election.cc:302] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b0a233dde4dd408281418532370591ed; no voters: 58a1f1fb5fe944d28f402fe090706160, 7c809573d8a6460498758e2ecaad1075
I20260811 14:34:27.670485 11768 raft_consensus.cc:2726] T 71d3c66a037f4d38bcfdb0bd113c160a P b0a233dde4dd408281418532370591ed [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 14:34:27.743014 11769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:27.742761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:34303)
0811 14:34:27.742908 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:27.742949 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":83,"threads_started":1}
I20260811 14:34:28.512766 11771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:28.512465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:52517)
0811 14:34:28.512624 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:28.512666 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260811 14:34:28.612954 11772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:28.612654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:58495)
0811 14:34:28.612856 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:28.612902 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":111,"threads_started":1}
I20260811 14:34:28.743830 11774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:28.743573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:49325)
0811 14:34:28.743736 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:28.743782 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":99,"threads_started":1}
I20260811 14:34:29.513504 11775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:29.513276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:46015)
0811 14:34:29.513412 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:29.513453 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":67,"threads_started":1}
I20260811 14:34:29.613781 11776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:29.613535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:57807)
0811 14:34:29.613684 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:29.613732 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":69,"threads_started":1}
I20260811 14:34:29.744640 11778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:29.744377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:57707)
0811 14:34:29.744534 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:29.744581 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260811 14:34:30.514287 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:30.514044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:48041)
0811 14:34:30.514195 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:30.514236 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":71,"threads_started":1}
I20260811 14:34:30.614609 11780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:30.614347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:46605)
0811 14:34:30.614509 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:30.614558 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260811 14:34:30.745512 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:30.745151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:33403)
0811 14:34:30.745364 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:30.745455 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":109,"threads_started":1}
I20260811 14:34:31.515218 11782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:31.514948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:35381)
0811 14:34:31.515126 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:31.515172 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":72,"threads_started":1}
I20260811 14:34:31.615609 11783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:31.615342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:36441)
0811 14:34:31.615511 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:31.615559 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":83,"threads_started":1}
I20260811 14:34:31.746392 11784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:31.746157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:51305)
0811 14:34:31.746296 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:31.746338 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260811 14:34:32.516172 11785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:32.515903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:52377)
0811 14:34:32.516071 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:32.516121 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":72,"threads_started":1}
I20260811 14:34:32.616303 11786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:32.616055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:58783)
0811 14:34:32.616215 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:32.616254 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":74,"threads_started":1}
I20260811 14:34:32.747172 11787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:32.746907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:39363)
0811 14:34:32.747070 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:32.747121 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260811 14:34:33.517222 11789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:33.516917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.193:60727)
0811 14:34:33.517132 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:33.517174 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":139,"threads_started":1}
I20260811 14:34:33.617367 11790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:33.617005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.194:43077)
0811 14:34:33.617214 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:33.617288 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":66,"threads_started":1}
I20260811 14:34:33.745035 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 11247
I20260811 14:34:33.749469 11791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:33.747940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38929 (local address 127.23.186.195:35571)
0811 14:34:33.749355 (+ 1415us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:33.749405 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.254:38929: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1392,"thread_start_us":296,"threads_started":1}
I20260811 14:34:33.753880 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 11440
I20260811 14:34:33.762055 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 11588
2026-08-11T14:34:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59465 ms)
[----------] 30 tests from SecurityITest (256834 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:33 dist-test-slave-cbsf krb5kdc[11794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:33 dist-test-slave-cbsf krb5kdc[11794](info): set up 2 sockets
Aug 11 14:34:33 dist-test-slave-cbsf krb5kdc[11794](info): commencing operation
krb5kdc: starting...
W20260811 14:34:34.900848 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:34 dist-test-slave-cbsf krb5kdc[11794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:34:35.041348 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38551
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:38551
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:35.128129 11810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:35.128418 11810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:35.128463 11810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:35.129673 11810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:35.129752 11810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:35.129776 11810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:35.129799 11810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:35.129829 11810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:35.131362 11810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:38551
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:38551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:35.131790 11810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:35.132056 11810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:35.137697 11820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:35.137703 11810 server_base.cc:1034] running on GCE node
W20260811 14:34:35.137697 11816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.137703 11815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:35.138327 11810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:35.138604 11810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:35.139751 11810 hybrid_clock.cc:648] HybridClock initialized: now 1786458875139741 us; error 35 us; skew 500 ppm
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:35.142525 11810 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:34:35.143079 11810 webserver.cc:460] Webserver started at http://127.23.186.254:45127/ using document root <none> and password file <none>
I20260811 14:34:35.143353 11810 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:35.143428 11810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:35.143587 11810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:35.144796 11810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "bfc7320300c64a6b8baaa76b159eb29f"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "1910c00b8a16d376255db3933b95477c"
server_key_iv: "4b152d01391ceaa33471334b2dcedf89"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.145181 11810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "bfc7320300c64a6b8baaa76b159eb29f"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "1910c00b8a16d376255db3933b95477c"
server_key_iv: "4b152d01391ceaa33471334b2dcedf89"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.146695 11810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:34:35.147949 11827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.148155 11810 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:35.148280 11810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "bfc7320300c64a6b8baaa76b159eb29f"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "1910c00b8a16d376255db3933b95477c"
server_key_iv: "4b152d01391ceaa33471334b2dcedf89"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.148378 11810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:35.189612 11810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:35.192173 11810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:35.192364 11810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:35.197242 11810 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:38551
I20260811 14:34:35.197286 11886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:38551 every 8 connection(s)
I20260811 14:34:35.197703 11810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:35.198392 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 11810
I20260811 14:34:35.198513 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:35.198753 24299 external_mini_cluster.cc:1428] Setting key 333aea21a03cf95c0f7799b911bf6d56
I20260811 14:34:35.198879 11887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:35.202545 11887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f: Bootstrap starting.
I20260811 14:34:35.203279 11887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:35.203625 11887 log.cc:824] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:35.204389 11887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f: No bootstrap required, opened a new log
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:35.206637 11887 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc7320300c64a6b8baaa76b159eb29f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38551 } }
I20260811 14:34:35.206826 11887 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:35.206880 11887 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc7320300c64a6b8baaa76b159eb29f, State: Initialized, Role: FOLLOWER
I20260811 14:34:35.207021 11887 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc7320300c64a6b8baaa76b159eb29f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38551 } }
I20260811 14:34:35.207101 11887 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:35.207126 11887 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:35.207177 11887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:35.208024 11887 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc7320300c64a6b8baaa76b159eb29f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38551 } }
I20260811 14:34:35.208160 11887 leader_election.cc:302] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc7320300c64a6b8baaa76b159eb29f; no voters:
I20260811 14:34:35.208362 11887 leader_election.cc:288] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:35.208448 11894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:35.208612 11894 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [term 1 LEADER]: Becoming Leader. State: Replica: bfc7320300c64a6b8baaa76b159eb29f, State: Running, Role: LEADER
I20260811 14:34:35.208732 11887 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:35.208763 11894 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc7320300c64a6b8baaa76b159eb29f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38551 } }
I20260811 14:34:35.209196 11896 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc7320300c64a6b8baaa76b159eb29f. Latest consensus state: current_term: 1 leader_uuid: "bfc7320300c64a6b8baaa76b159eb29f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc7320300c64a6b8baaa76b159eb29f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38551 } } }
I20260811 14:34:35.209290 11896 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:35.209527 11904 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:35.210022 11897 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfc7320300c64a6b8baaa76b159eb29f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc7320300c64a6b8baaa76b159eb29f" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 38551 } } }
I20260811 14:34:35.210228 11897 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:35.210742 11904 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:35.211926 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.199364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33176 (local address 127.23.186.254:38551)
0811 14:34:35.199565 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.199573 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.199611 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:34:35.200095 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.200104 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.200153 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.200273 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:35.201154 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.202066 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.203235 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.204024 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.204355 (+ 331us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.207396 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:35.207401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:35.207401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:35.207415 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:35.209825 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.210295 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.210299 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.210300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.210414 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.210664 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.210664 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.210665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.211012 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:35.211169 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.211571 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.211725 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
I20260811 14:34:35.212816 11904 catalog_manager.cc:1347] Generated new cluster ID: a23688406f4d4506a88f1967f6c973a2
I20260811 14:34:35.212867 11904 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:35.228747 11904 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:35.229594 11904 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:35.232432 11904 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f: Generated new TSK 0
I20260811 14:34:35.232635 11904 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:34:35.267613 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38551
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:35.354660 11919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:35.354912 11919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:35.354952 11919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:35.356132 11919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:35.356205 11919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:35.356251 11919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:35.357945 11919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:35.358242 11919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:35.358500 11919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:35.363871 11928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.363837 11926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.363843 11925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:35.364203 11919 server_base.cc:1034] running on GCE node
I20260811 14:34:35.364455 11919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:35.364761 11919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:35.365948 11919 hybrid_clock.cc:648] HybridClock initialized: now 1786458875365940 us; error 44 us; skew 500 ppm
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:35.369079 11919 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:34:35.369581 11919 webserver.cc:460] Webserver started at http://127.23.186.193:39215/ using document root <none> and password file <none>
I20260811 14:34:35.369837 11919 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:35.369911 11919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:35.370069 11919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:35.371209 11919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "c06a3cac1827403d8135bc43ce07252f"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "2aa525ab85224784383d464d22478a53"
server_key_iv: "335ad67ab0df2f128dffe6e4c42c9712"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.371606 11919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "c06a3cac1827403d8135bc43ce07252f"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "2aa525ab85224784383d464d22478a53"
server_key_iv: "335ad67ab0df2f128dffe6e4c42c9712"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.373173 11919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:35.374090 11936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.374315 11919 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:35.374464 11919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "c06a3cac1827403d8135bc43ce07252f"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "2aa525ab85224784383d464d22478a53"
server_key_iv: "335ad67ab0df2f128dffe6e4c42c9712"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.374550 11919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:35.389493 11919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:35.391827 11919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:35.392025 11919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:35.392297 11919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:35.392647 11919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:35.392729 11919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.392797 11919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:35.392829 11919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.398936 11919 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:33281
I20260811 14:34:35.398977 12061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:33281 every 8 connection(s)
I20260811 14:34:35.399325 11919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:35.404644 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 11919
I20260811 14:34:35.404822 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:34:35.405036 24299 external_mini_cluster.cc:1428] Setting key 008f0f81af086dae12176c67086da079
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:35.409705 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.400157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:50907 (local address 127.23.186.254:38551)
0811 14:34:35.400227 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.400228 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.400412 (+ 184us) server_negotiation.cc:407] Connection header received
0811 14:34:35.401010 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.401011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.401022 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.401085 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:35.402096 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.402629 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.403628 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.403826 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.403884 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.406413 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:35.406415 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:35.406415 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:35.406423 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:35.407998 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.408525 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.408526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.408527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.408630 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.408961 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.408961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.408962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.409076 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:35.409127 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.409483 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.409603 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:34:35.409955 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.400091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38551 (local address 127.23.186.193:50907)
0811 14:34:35.400273 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.400317 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.400904 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.401161 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.401165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.401768 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:35.401981 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.401985 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.402721 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.402721 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.403544 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.403546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.403905 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.403905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.404021 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.404610 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:35.404618 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:35.406303 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:35.408088 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:35.408097 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:35.408100 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:35.408438 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:35.408725 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:35.408726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:35.408727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:35.408880 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:35.409142 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:35.409149 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:35.409334 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.409444 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.409634 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":104,"threads_started":1}
I20260811 14:34:35.410475 12062 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38551
I20260811 14:34:35.410622 12062 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:35.410820 12062 heartbeater.cc:511] Master 127.23.186.254:38551 requested a full tablet report, sending...
I20260811 14:34:35.411367 11845 ts_manager.cc:194] Registered new tserver with Master: c06a3cac1827403d8135bc43ce07252f (127.23.186.193:33281)
I20260811 14:34:35.412025 11845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:50907
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:34:35.433849 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38551
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:35.519934 12067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:35.520206 12067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:35.520249 12067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:35.521482 12067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:35.521566 12067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:35.521615 12067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:35.523165 12067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:35.523489 12067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:35.523751 12067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:35.529036 12074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.529035 12076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.529081 12073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:35.529307 12067 server_base.cc:1034] running on GCE node
I20260811 14:34:35.529565 12067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:35.529845 12067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:35.531030 12067 hybrid_clock.cc:648] HybridClock initialized: now 1786458875531011 us; error 32 us; skew 500 ppm
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:35.533967 12067 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:34:35.534466 12067 webserver.cc:460] Webserver started at http://127.23.186.194:42969/ using document root <none> and password file <none>
I20260811 14:34:35.534713 12067 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:35.534788 12067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:35.534950 12067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:35.536099 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "8b3ef2baa975447c88a678b9cf145729"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "b2141c318ea42db71703267309783742"
server_key_iv: "4720d5588f33d6b99bb0d2bcdc0d84d8"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.536494 12067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "8b3ef2baa975447c88a678b9cf145729"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "b2141c318ea42db71703267309783742"
server_key_iv: "4720d5588f33d6b99bb0d2bcdc0d84d8"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.538048 12067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:35.538885 12083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.539206 12067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:35.539324 12067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "8b3ef2baa975447c88a678b9cf145729"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "b2141c318ea42db71703267309783742"
server_key_iv: "4720d5588f33d6b99bb0d2bcdc0d84d8"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.539408 12067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:35.555464 12067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:35.558223 12067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:35.558418 12067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:35.558701 12067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:35.559042 12067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:35.559099 12067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.559139 12067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:35.559187 12067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.565800 12067 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:37403
I20260811 14:34:35.565850 12211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:37403 every 8 connection(s)
I20260811 14:34:35.566212 12067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:34:35.570309 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 12067
I20260811 14:34:35.570415 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:34:35.570644 24299 external_mini_cluster.cc:1428] Setting key 983e361ba48e079d3d290c5923521d68
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:35.577042 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.567113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:45735 (local address 127.23.186.254:38551)
0811 14:34:35.567184 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.567186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.567393 (+ 207us) server_negotiation.cc:407] Connection header received
0811 14:34:35.567887 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.567888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.567898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.567963 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:35.569036 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.569558 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.570777 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.571055 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.571158 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.573795 (+ 2637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:35.573797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:35.573798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:35.573808 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:35.575335 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.575917 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.575918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.575919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.575975 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.576289 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.576290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.576290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.576414 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:35.576469 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.576887 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.576972 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:34:35.577385 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.567079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38551 (local address 127.23.186.194:45735)
0811 14:34:35.567244 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.567295 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.567797 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.568017 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.568020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.568637 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:35.568936 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.568942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.569657 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.569658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.570636 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.570638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.571171 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.571173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.571315 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.571908 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:35.571920 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:35.573677 (+ 1757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:35.575446 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:35.575457 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:35.575460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:35.575826 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:35.576050 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:35.576051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:35.576051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:35.576213 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:35.576493 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:35.576501 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:35.576729 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.576870 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.577080 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260811 14:34:35.578101 12212 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38551
I20260811 14:34:35.578238 12212 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:35.578491 12212 heartbeater.cc:511] Master 127.23.186.254:38551 requested a full tablet report, sending...
I20260811 14:34:35.579113 11845 ts_manager.cc:194] Registered new tserver with Master: 8b3ef2baa975447c88a678b9cf145729 (127.23.186.194:37403)
I20260811 14:34:35.579632 11845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:45735
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:34:35.600255 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38551
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:38893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:35.687227 12218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:35.687462 12218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:35.687489 12218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:35.688810 12218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:35.688862 12218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:35.688918 12218 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:35.690533 12218 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:38893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:38551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12218
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:35.690845 12218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:35.691128 12218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:35.696841 12226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.696846 12228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:35.696863 12225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:35.697232 12218 server_base.cc:1034] running on GCE node
I20260811 14:34:35.697467 12218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:35.697705 12218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:35.698855 12218 hybrid_clock.cc:648] HybridClock initialized: now 1786458875698835 us; error 35 us; skew 500 ppm
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:35.701524 12218 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:34:35.702066 12218 webserver.cc:460] Webserver started at http://127.23.186.195:41049/ using document root <none> and password file <none>
I20260811 14:34:35.702311 12218 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:35.702381 12218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:35.702540 12218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:35.703658 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "a6b08e0c299d131202aa9afc317f664b"
server_key_iv: "64df13c6d5d2989fccba3c53bcbc2b45"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.704039 12218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "a6b08e0c299d131202aa9afc317f664b"
server_key_iv: "64df13c6d5d2989fccba3c53bcbc2b45"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.705600 12218 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:35.706519 12236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.706720 12218 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:35.706838 12218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934"
format_stamp: "Formatted at 2026-08-11 14:34:35 on dist-test-slave-cbsf"
server_key: "a6b08e0c299d131202aa9afc317f664b"
server_key_iv: "64df13c6d5d2989fccba3c53bcbc2b45"
server_key_version: "encryptionkey@0"
I20260811 14:34:35.706921 12218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:35.722358 12218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:35.724301 12218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:35.724468 12218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:35.724802 12218 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:35.725142 12218 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:35.725198 12218 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.725240 12218 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:35.725267 12218 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:35.731503 12218 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:36811
I20260811 14:34:35.731549 12367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:36811 every 8 connection(s)
I20260811 14:34:35.731899 12218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:35.736809 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 12218
I20260811 14:34:35.736908 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:34:35.737174 24299 external_mini_cluster.cc:1428] Setting key 8c9aa42603b739382880b0d61b554c61
Aug 11 14:34:35 dist-test-slave-cbsf krb5kdc[11794](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458875, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:35.742333 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.732930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:34463 (local address 127.23.186.254:38551)
0811 14:34:35.732982 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.732984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.733217 (+ 233us) server_negotiation.cc:407] Connection header received
0811 14:34:35.733696 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.733697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.733708 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.733760 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:35.734945 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.735484 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.736519 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.736833 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.736955 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.739531 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:35.739533 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:35.739533 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:35.739540 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:35.741088 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.741481 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.741482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.741482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.741537 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.741764 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.741764 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.741764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.741878 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:35.741930 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.742196 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.742264 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:34:35.742611 12370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.732913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:38551 (local address 127.23.186.195:34463)
0811 14:34:35.733082 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.733126 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.733612 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.733816 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.733819 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.734538 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:35.734812 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.734819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.735580 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.735581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.736424 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.736426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.737000 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.737001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.737186 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.737831 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:35.737844 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:35.739439 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:35.741169 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:35.741176 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:35.741178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:35.741411 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:35.741596 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:35.741597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:35.741597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:35.741708 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:35.741935 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:35.741940 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:35.742070 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.742166 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.742302 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":96,"threads_started":1}
I20260811 14:34:35.743124 12368 heartbeater.cc:347] Connected to a master server at 127.23.186.254:38551
I20260811 14:34:35.743237 12368 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:35.743458 12368 heartbeater.cc:511] Master 127.23.186.254:38551 requested a full tablet report, sending...
I20260811 14:34:35.743872 11845 ts_manager.cc:194] Registered new tserver with Master: f2f82c5b1de246c0bdc4ad99e5dc9934 (127.23.186.195:36811)
I20260811 14:34:35.744277 11845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:34463
I20260811 14:34:35.749248 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:34:35.757211 11891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.750734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33184 (local address 127.23.186.254:38551)
0811 14:34:35.750823 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.750826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.750920 (+ 94us) server_negotiation.cc:407] Connection header received
0811 14:34:35.751012 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.751013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.751025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.751079 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:35.751777 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.752288 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.753156 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.753353 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.753432 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.754680 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:35.754682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:35.754682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:35.754690 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:35.755851 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.756238 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.756239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.756240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.756306 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:35.756520 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:35.756521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:35.756521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:35.756768 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:35.756867 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.756983 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.757081 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 14:34:35.758942 11845 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33184:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:34:35.760556 11845 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:34:35.767540 12386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.763037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:37403 (local address 127.0.0.1:60428)
0811 14:34:35.763587 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.763625 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.763784 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.764148 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.764149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.764152 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:35.764202 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.764204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.765202 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.765202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.766366 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.766369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.767187 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.767188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.767277 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.767293 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.767383 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.767447 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":55,"threads_started":1}
I20260811 14:34:35.767552 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.763149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60428 (local address 127.23.186.194:37403)
0811 14:34:35.763351 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.763357 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.763735 (+ 378us) server_negotiation.cc:407] Connection header received
0811 14:34:35.763904 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.763906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.763924 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.763983 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:35.764294 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.765101 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.766459 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.767077 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.767174 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.767238 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.767414 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.767459 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260811 14:34:35.767894 12381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.763059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36811 (local address 127.0.0.1:42480)
0811 14:34:35.763587 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.763624 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.763720 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.764024 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.764028 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.764030 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:35.764110 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.764112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.765577 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.765578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.766796 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.766799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.767700 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.767701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.767768 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.767772 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.767802 (+ 30us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.767845 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":461,"threads_started":1}
I20260811 14:34:35.768308 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.763059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:33281 (local address 127.0.0.1:37548)
0811 14:34:35.763868 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.763879 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.763923 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.764175 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.764175 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.764177 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:35.764229 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.764230 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.765560 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.765561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.766797 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.766799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.767990 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.767991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.768149 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.768154 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.768209 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.768246 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"mutex_wait_us":47,"thread_start_us":474,"threads_started":1}
I20260811 14:34:35.768318 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.763255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.23.186.195:36811)
0811 14:34:35.763405 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.763409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.763703 (+ 294us) server_negotiation.cc:407] Connection header received
0811 14:34:35.763825 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.763828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.763848 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.763931 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:35.764197 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.765439 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.766910 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.767583 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.768081 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.768153 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.768182 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.768232 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260811 14:34:35.768509 12139 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5999f0a08d41f1b0db6c6031295171 (DEFAULT_TABLE table=test-table [id=a29eecd0b7b24e449fc79fbe4444a06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:35.768651 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.763268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37548 (local address 127.23.186.193:33281)
0811 14:34:35.763445 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.763449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.763988 (+ 539us) server_negotiation.cc:407] Connection header received
0811 14:34:35.764037 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.764039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.764054 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.764181 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:35.764295 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.765448 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.766901 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.767660 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.767977 (+ 317us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.768045 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.768497 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.768539 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":112,"threads_started":1}
I20260811 14:34:35.768991 12139 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5999f0a08d41f1b0db6c6031295171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:35.769026 12290 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5999f0a08d41f1b0db6c6031295171 (DEFAULT_TABLE table=test-table [id=a29eecd0b7b24e449fc79fbe4444a06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:35.769374 12290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5999f0a08d41f1b0db6c6031295171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:35.769343 11987 tablet_service.cc:1467] Processing CreateTablet for tablet 2e5999f0a08d41f1b0db6c6031295171 (DEFAULT_TABLE table=test-table [id=a29eecd0b7b24e449fc79fbe4444a06c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:35.769675 11987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e5999f0a08d41f1b0db6c6031295171. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:35.771958 12387 tablet_bootstrap.cc:492] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Bootstrap starting.
I20260811 14:34:35.772938 12388 tablet_bootstrap.cc:492] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Bootstrap starting.
I20260811 14:34:35.773326 12389 tablet_bootstrap.cc:492] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: Bootstrap starting.
I20260811 14:34:35.773343 12387 tablet_bootstrap.cc:654] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:35.773689 12388 tablet_bootstrap.cc:654] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:35.773958 12389 tablet_bootstrap.cc:654] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:35.774029 12387 log.cc:824] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:35.774288 12388 log.cc:824] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:35.774575 12389 log.cc:824] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:35.774950 12387 tablet_bootstrap.cc:492] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: No bootstrap required, opened a new log
I20260811 14:34:35.775049 12387 ts_tablet_manager.cc:1397] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:34:35.775348 12388 tablet_bootstrap.cc:492] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: No bootstrap required, opened a new log
I20260811 14:34:35.775406 12388 ts_tablet_manager.cc:1397] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:34:35.775646 12389 tablet_bootstrap.cc:492] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: No bootstrap required, opened a new log
I20260811 14:34:35.775718 12389 ts_tablet_manager.cc:1397] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:34:35.776963 12388 raft_consensus.cc:348] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.777199 12388 raft_consensus.cc:374] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:35.777252 12388 raft_consensus.cc:729] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f82c5b1de246c0bdc4ad99e5dc9934, State: Initialized, Role: FOLLOWER
I20260811 14:34:35.777261 12387 raft_consensus.cc:348] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.777386 12388 consensus_queue.cc:260] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.777441 12387 raft_consensus.cc:374] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:35.777484 12387 raft_consensus.cc:729] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3ef2baa975447c88a678b9cf145729, State: Initialized, Role: FOLLOWER
I20260811 14:34:35.777601 12387 consensus_queue.cc:260] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.777696 12388 ts_tablet_manager.cc:1428] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 14:34:35.777817 12368 heartbeater.cc:503] Master 127.23.186.254:38551 was elected leader, sending a full tablet report...
I20260811 14:34:35.777901 12387 ts_tablet_manager.cc:1428] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:35.778016 12212 heartbeater.cc:503] Master 127.23.186.254:38551 was elected leader, sending a full tablet report...
I20260811 14:34:35.777973 12389 raft_consensus.cc:348] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.778146 12389 raft_consensus.cc:374] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:35.778188 12389 raft_consensus.cc:729] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c06a3cac1827403d8135bc43ce07252f, State: Initialized, Role: FOLLOWER
I20260811 14:34:35.778285 12389 consensus_queue.cc:260] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.778570 12389 ts_tablet_manager.cc:1428] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:35.778856 12062 heartbeater.cc:503] Master 127.23.186.254:38551 was elected leader, sending a full tablet report...
W20260811 14:34:35.816910 12214 tablet.cc:2369] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:34:35.900136 12063 tablet.cc:2369] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:34:35.920236 12396 raft_consensus.cc:482] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:35.920384 12396 raft_consensus.cc:504] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.920778 12396 leader_election.cc:288] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f82c5b1de246c0bdc4ad99e5dc9934 (127.23.186.195:36811), c06a3cac1827403d8135bc43ce07252f (127.23.186.193:33281)
I20260811 14:34:35.924189 12383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.921240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:35523 (local address 127.23.186.195:36811)
0811 14:34:35.921336 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.921338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.921348 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:35.921374 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.921375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.921385 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.921445 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:35.921598 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.922380 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.923380 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.923802 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.923862 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.923895 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.924060 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.924113 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 14:34:35.924198 12215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.920999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:36811 (local address 127.23.186.194:35523)
0811 14:34:35.921049 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.921063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.921130 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.921463 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.921464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.921465 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:35.921528 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.921530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.922490 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.922491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.923276 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.923278 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.923891 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.923893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.923988 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.923991 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.924034 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.924067 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 14:34:35.924245 12384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.921099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:52463 (local address 127.23.186.193:33281)
0811 14:34:35.921191 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.921194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.921324 (+ 130us) server_negotiation.cc:407] Connection header received
0811 14:34:35.921364 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.921365 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.921377 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.921445 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:35.921694 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.922388 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.923435 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.923811 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.923910 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.923919 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.924126 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.924169 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 14:34:35.924394 12398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.920999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:33281 (local address 127.23.186.194:52463)
0811 14:34:35.921247 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.921256 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:35.921314 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:35.921589 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:35.921590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:35.921590 (+ 0us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:35.921632 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.921632 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.922514 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.922516 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.923295 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:35.923297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.923937 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.923938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:35.924014 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.924038 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 14:34:35.924096 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 14:34:35.924142 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":27,"thread_start_us":137,"threads_started":1}
I20260811 14:34:35.924603 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5999f0a08d41f1b0db6c6031295171" candidate_uuid: "8b3ef2baa975447c88a678b9cf145729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" is_pre_election: true
I20260811 14:34:35.924829 12312 raft_consensus.cc:2454] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b3ef2baa975447c88a678b9cf145729 in term 0.
I20260811 14:34:35.924800 12009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5999f0a08d41f1b0db6c6031295171" candidate_uuid: "8b3ef2baa975447c88a678b9cf145729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06a3cac1827403d8135bc43ce07252f" is_pre_election: true
I20260811 14:34:35.924957 12009 raft_consensus.cc:2454] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b3ef2baa975447c88a678b9cf145729 in term 0.
I20260811 14:34:35.925094 12086 leader_election.cc:302] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b3ef2baa975447c88a678b9cf145729, f2f82c5b1de246c0bdc4ad99e5dc9934; no voters:
I20260811 14:34:35.925278 12396 raft_consensus.cc:2781] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:34:35.925339 12396 raft_consensus.cc:482] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:34:35.925357 12396 raft_consensus.cc:3037] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:35.926141 12396 raft_consensus.cc:504] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.926301 12396 leader_election.cc:288] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [CANDIDATE]: Term 1 election: Requested vote from peers f2f82c5b1de246c0bdc4ad99e5dc9934 (127.23.186.195:36811), c06a3cac1827403d8135bc43ce07252f (127.23.186.193:33281)
I20260811 14:34:35.926512 12312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5999f0a08d41f1b0db6c6031295171" candidate_uuid: "8b3ef2baa975447c88a678b9cf145729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934"
I20260811 14:34:35.926522 12009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e5999f0a08d41f1b0db6c6031295171" candidate_uuid: "8b3ef2baa975447c88a678b9cf145729" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06a3cac1827403d8135bc43ce07252f"
I20260811 14:34:35.926604 12312 raft_consensus.cc:3037] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:35.926604 12009 raft_consensus.cc:3037] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:35.927528 12312 raft_consensus.cc:2454] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b3ef2baa975447c88a678b9cf145729 in term 1.
I20260811 14:34:35.927528 12009 raft_consensus.cc:2454] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b3ef2baa975447c88a678b9cf145729 in term 1.
I20260811 14:34:35.927729 12087 leader_election.cc:302] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8b3ef2baa975447c88a678b9cf145729, c06a3cac1827403d8135bc43ce07252f; no voters:
I20260811 14:34:35.927834 12396 raft_consensus.cc:2781] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:35.928015 12396 raft_consensus.cc:686] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 1 LEADER]: Becoming Leader. State: Replica: 8b3ef2baa975447c88a678b9cf145729, State: Running, Role: LEADER
I20260811 14:34:35.928128 12396 consensus_queue.cc:237] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } }
I20260811 14:34:35.929040 11845 catalog_manager.cc:5495] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b3ef2baa975447c88a678b9cf145729 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "8b3ef2baa975447c88a678b9cf145729" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3ef2baa975447c88a678b9cf145729" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 37403 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 } health_report { overall_health: UNKNOWN } } }
I20260811 14:34:35.950110 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.947616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60440 (local address 127.23.186.194:37403)
0811 14:34:35.947679 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:35.947681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:35.947690 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:34:35.947751 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:35.947752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:35.947773 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:35.947837 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:34:35.947962 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.948436 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.949290 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:35.949495 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:35.949558 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:35.949704 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:34:35.949841 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:34:35.949907 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:35.949943 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 14:34:35.949970 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 14:34:35.952335 12312 raft_consensus.cc:1261] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 1 FOLLOWER]: Refusing update from remote peer 8b3ef2baa975447c88a678b9cf145729: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:35.952374 12009 raft_consensus.cc:1261] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 1 FOLLOWER]: Refusing update from remote peer 8b3ef2baa975447c88a678b9cf145729: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:35.952803 12396 consensus_queue.cc:1035] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06a3cac1827403d8135bc43ce07252f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:35.952975 12399 consensus_queue.cc:1035] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f82c5b1de246c0bdc4ad99e5dc9934" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 36811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:35.956197 12402 mvcc.cc:205] Tried to move back new op lower bound from 7317335555896983552 to 7317335555802157056. Current Snapshot: MvccSnapshot[applied={T|T < 7317335555896983552}]
I20260811 14:34:35.958794 11844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:35.958911 11844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33184:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:35.959726 11844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfc7320300c64a6b8baaa76b159eb29f: Sending DeleteTablet for 3 replicas of tablet 2e5999f0a08d41f1b0db6c6031295171
I20260811 14:34:35.960062 12139 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5999f0a08d41f1b0db6c6031295171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:35 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:60428
I20260811 14:34:35.960096 11987 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5999f0a08d41f1b0db6c6031295171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:35 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:37548
I20260811 14:34:35.960155 12290 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e5999f0a08d41f1b0db6c6031295171 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:35 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:42480
I20260811 14:34:35.960327 12413 tablet_replica.cc:330] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f: stopping tablet replica
I20260811 14:34:35.960489 12413 raft_consensus.cc:2229] T 2e5999f0a08d41f1b0db6c6031295171 P c06a3cac1827403d8135bc43ce07252f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:35.960594 12414 tablet_replica.cc:330] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: stopping tablet replica
I20260811 14:34:35.960647 12412 tablet_replica.cc:330] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: stopping tablet replica
I20260811 14:34:35.960781 12412 raft_consensus.cc:2229] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:34:35.960795 12414 raft_consensus.cc:2229] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:35.960796 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 11919
I20260811 14:34:35.960918 12412 raft_consensus.cc:2258] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:35.962251 12414 raft_consensus.cc:2258] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:35.963344 12414 ts_tablet_manager.cc:1905] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:35.965389 12414 ts_tablet_manager.cc:1918] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:35.965481 12414 log.cc:1196] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/2e5999f0a08d41f1b0db6c6031295171
I20260811 14:34:35.965852 12414 ts_tablet_manager.cc:1939] T 2e5999f0a08d41f1b0db6c6031295171 P f2f82c5b1de246c0bdc4ad99e5dc9934: Deleting consensus metadata
I20260811 14:34:35.966459 12412 ts_tablet_manager.cc:1905] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:35.966500 11829 catalog_manager.cc:4841] TS f2f82c5b1de246c0bdc4ad99e5dc9934 (127.23.186.195:36811): tablet 2e5999f0a08d41f1b0db6c6031295171 (table test-table [id=a29eecd0b7b24e449fc79fbe4444a06c]) successfully deleted
I20260811 14:34:35.968430 12412 ts_tablet_manager.cc:1918] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:35.968525 12412 log.cc:1196] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/2e5999f0a08d41f1b0db6c6031295171
I20260811 14:34:35.968863 12412 ts_tablet_manager.cc:1939] T 2e5999f0a08d41f1b0db6c6031295171 P 8b3ef2baa975447c88a678b9cf145729: Deleting consensus metadata
I20260811 14:34:35.969380 11828 catalog_manager.cc:4841] TS 8b3ef2baa975447c88a678b9cf145729 (127.23.186.194:37403): tablet 2e5999f0a08d41f1b0db6c6031295171 (table test-table [id=a29eecd0b7b24e449fc79fbe4444a06c]) successfully deleted
W20260811 14:34:35.971706 11830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:33281 (error 108)
I20260811 14:34:35.972334 12385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:35.972180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:33281 (local address 127.0.0.1:37552)
0811 14:34:35.972248 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:35.972297 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:33281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 14:34:35.972405 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 12067
W20260811 14:34:35.976886 11830 catalog_manager.cc:4584] TS c06a3cac1827403d8135bc43ce07252f (127.23.186.193:33281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2e5999f0a08d41f1b0db6c6031295171: Network error: Client connection negotiation failed: client connection to 127.23.186.193:33281: connect: Connection refused (error 111)
I20260811 14:34:35.981423 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 12218
I20260811 14:34:35.989230 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 11810
2026-08-11T14:34:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2228 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T14:34:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:36Z Disabled control of system clock
I20260811 14:34:36.022925 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41343
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:41343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:36.106169 12419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:36.106412 12419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:36.106433 12419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:36.107525 12419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:36.107563 12419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:36.107575 12419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:36.107589 12419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:36.107600 12419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:36.108894 12419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:41343
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:36.109125 12419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:36.109371 12419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:36.114420 12428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.114420 12425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.114408 12426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:36.114711 12419 server_base.cc:1034] running on GCE node
I20260811 14:34:36.114984 12419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:36.115296 12419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:36.116464 12419 hybrid_clock.cc:648] HybridClock initialized: now 1786458876116459 us; error 40 us; skew 500 ppm
I20260811 14:34:36.117787 12419 webserver.cc:460] Webserver started at http://127.23.186.254:33949/ using document root <none> and password file <none>
I20260811 14:34:36.118041 12419 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:36.118121 12419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:36.118273 12419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:36.119407 12419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "9c99e9bda98449eaad327ed16ee2c160"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "620cf3db9bed8107424f618173f2af60"
server_key_iv: "006dbf1e24672244f36341b3911b88ea"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.119788 12419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "9c99e9bda98449eaad327ed16ee2c160"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "620cf3db9bed8107424f618173f2af60"
server_key_iv: "006dbf1e24672244f36341b3911b88ea"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.121289 12419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:36.122063 12435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.122234 12419 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:34:36.122346 12419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "9c99e9bda98449eaad327ed16ee2c160"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "620cf3db9bed8107424f618173f2af60"
server_key_iv: "006dbf1e24672244f36341b3911b88ea"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.122423 12419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:36.154774 12419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:36.155169 12419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:36.155333 12419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:36.159874 12419 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:41343
I20260811 14:34:36.159940 12493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:41343 every 8 connection(s)
I20260811 14:34:36.160336 12419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:36.161500 12494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:36.163894 12494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: Bootstrap starting.
I20260811 14:34:36.164484 12494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:36.164832 12494 log.cc:824] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:36.165588 12494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: No bootstrap required, opened a new log
I20260811 14:34:36.167207 12494 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c99e9bda98449eaad327ed16ee2c160" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41343 } }
I20260811 14:34:36.167361 12494 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:36.167384 12494 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c99e9bda98449eaad327ed16ee2c160, State: Initialized, Role: FOLLOWER
I20260811 14:34:36.167554 12494 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c99e9bda98449eaad327ed16ee2c160" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41343 } }
I20260811 14:34:36.167644 12494 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:36.167668 12494 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:36.167698 12494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:36.168238 12494 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c99e9bda98449eaad327ed16ee2c160" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41343 } }
I20260811 14:34:36.168248 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 12419
I20260811 14:34:36.168332 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:36.168373 12494 leader_election.cc:302] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c99e9bda98449eaad327ed16ee2c160; no voters:
I20260811 14:34:36.168555 24299 external_mini_cluster.cc:1428] Setting key 4826d9f1b1c7ab2d68654bab59d8854a
I20260811 14:34:36.168596 12494 leader_election.cc:288] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:36.168632 12498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:36.168833 12498 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [term 1 LEADER]: Becoming Leader. State: Replica: 9c99e9bda98449eaad327ed16ee2c160, State: Running, Role: LEADER
I20260811 14:34:36.168977 12494 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:36.169108 12498 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c99e9bda98449eaad327ed16ee2c160" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41343 } }
I20260811 14:34:36.169709 12500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c99e9bda98449eaad327ed16ee2c160. Latest consensus state: current_term: 1 leader_uuid: "9c99e9bda98449eaad327ed16ee2c160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c99e9bda98449eaad327ed16ee2c160" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41343 } } }
I20260811 14:34:36.169837 12500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:36.170271 12499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c99e9bda98449eaad327ed16ee2c160" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c99e9bda98449eaad327ed16ee2c160" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41343 } } }
I20260811 14:34:36.170418 12499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160 [sys.catalog]: This master's current role is: LEADER
W20260811 14:34:36.171098 12519 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:34:36.171159 12519 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:34:36.171203 12510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:36.171361 12510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:36.173125 12510 catalog_manager.cc:1347] Generated new cluster ID: 8f655111f76848eaaf08320bab6c22fe
I20260811 14:34:36.173185 12510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:36.174968 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.169139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.23.186.254:41343)
0811 14:34:36.169539 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.169549 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.169585 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:34:36.169734 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.169742 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.169818 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.170245 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:36.170361 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.171654 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.172747 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.173348 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.173485 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.174259 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:36.174263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:36.174263 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:36.174453 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:36.174492 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:36.174504 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:36.174589 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.174663 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.174747 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":84,"threads_started":1}
I20260811 14:34:36.185595 12510 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:36.186342 12510 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:36.193610 12510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: Generated new TSK 0
I20260811 14:34:36.193832 12510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:36.198659 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41343
--builtin_ntp_servers=127.23.186.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:36.307217 12522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:36.307581 12522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:36.307629 12522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:36.310011 12522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:36.310133 12522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:36.310225 12522 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:36.312047 12522 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:36.312348 12522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:36.312752 12522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:36.318082 12527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.318104 12530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.318079 12528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:36.318836 12522 server_base.cc:1034] running on GCE node
I20260811 14:34:36.319104 12522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:36.319419 12522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:36.320618 12522 hybrid_clock.cc:648] HybridClock initialized: now 1786458876320605 us; error 29 us; skew 500 ppm
I20260811 14:34:36.322243 12522 webserver.cc:460] Webserver started at http://127.23.186.193:42263/ using document root <none> and password file <none>
I20260811 14:34:36.322571 12522 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:36.322623 12522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:36.322767 12522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:36.323915 12522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "60911084a979477fbc5eca3a9c368f7f"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "8b248485104ad4eefb5957783441db68"
server_key_iv: "e4a3513f7a5c709ad4e2b1fd2d6ee7b1"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.324306 12522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "60911084a979477fbc5eca3a9c368f7f"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "8b248485104ad4eefb5957783441db68"
server_key_iv: "e4a3513f7a5c709ad4e2b1fd2d6ee7b1"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.325892 12522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:36.327013 12537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.327186 12522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:36.327275 12522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "60911084a979477fbc5eca3a9c368f7f"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "8b248485104ad4eefb5957783441db68"
server_key_iv: "e4a3513f7a5c709ad4e2b1fd2d6ee7b1"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.327361 12522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:36.343461 12522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:36.343853 12522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:36.344022 12522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:36.344326 12522 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:36.344782 12522 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:36.344841 12522 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.344887 12522 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:36.344914 12522 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.352159 12522 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:41203
I20260811 14:34:36.352201 12670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:41203 every 8 connection(s)
I20260811 14:34:36.352622 12522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:36.356563 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 12522
I20260811 14:34:36.356738 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:34:36.356959 24299 external_mini_cluster.cc:1428] Setting key a10eaeaf3a60fec4d1737d521e6bf142
I20260811 14:34:36.358064 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41343
--builtin_ntp_servers=127.23.186.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:34:36.358700 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.353512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:47003 (local address 127.23.186.254:41343)
0811 14:34:36.353586 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.353588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.353840 (+ 252us) server_negotiation.cc:407] Connection header received
0811 14:34:36.354390 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.354391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.354411 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.354481 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:36.355253 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.355808 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.357035 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.357315 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.357411 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.358413 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:36.358414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:36.358415 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:36.358420 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:36.358437 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:36.358442 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:36.358490 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.358572 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.358616 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:34:36.358925 12673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.353473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41343 (local address 127.23.186.193:47003)
0811 14:34:36.353702 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.353747 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.354288 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.354568 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.354572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.354581 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:36.355141 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.355146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.355923 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.355924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.356938 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.356941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.357425 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.357426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.357580 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.358233 (+ 653us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:36.358234 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:36.358281 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:36.358320 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:36.358490 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:36.358496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:36.358499 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.358577 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.358692 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":119,"threads_started":1}
I20260811 14:34:36.360258 12671 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41343
I20260811 14:34:36.360386 12671 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:36.360611 12671 heartbeater.cc:511] Master 127.23.186.254:41343 requested a full tablet report, sending...
I20260811 14:34:36.361223 12454 ts_manager.cc:194] Registered new tserver with Master: 60911084a979477fbc5eca3a9c368f7f (127.23.186.193:41203)
I20260811 14:34:36.361881 12454 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.193:47003
W20260811 14:34:36.444566 12674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:36.444870 12674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:36.444911 12674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:36.446102 12674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:36.446178 12674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:36.446223 12674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:36.447629 12674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:36.447925 12674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:36.448267 12674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:36.453687 12680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.453676 12681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.453676 12683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:36.454136 12674 server_base.cc:1034] running on GCE node
I20260811 14:34:36.454416 12674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:36.454742 12674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:36.455893 12674 hybrid_clock.cc:648] HybridClock initialized: now 1786458876455886 us; error 37 us; skew 500 ppm
I20260811 14:34:36.457422 12674 webserver.cc:460] Webserver started at http://127.23.186.194:45675/ using document root <none> and password file <none>
I20260811 14:34:36.457712 12674 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:36.457764 12674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:36.457943 12674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:36.459097 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "cc1a534daf1c4e349448eb6d025f7a65"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "083059b0edfc0e68264c273bda107eeb"
server_key_iv: "2154459d212594f75159b09128b26309"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.459568 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "cc1a534daf1c4e349448eb6d025f7a65"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "083059b0edfc0e68264c273bda107eeb"
server_key_iv: "2154459d212594f75159b09128b26309"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.461097 12674 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:36.461985 12689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.462225 12674 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:36.462324 12674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "cc1a534daf1c4e349448eb6d025f7a65"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "083059b0edfc0e68264c273bda107eeb"
server_key_iv: "2154459d212594f75159b09128b26309"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.462406 12674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:36.471141 12674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:36.471488 12674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:36.471652 12674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:36.471901 12674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:36.472254 12674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:36.472311 12674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.472355 12674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:36.472383 12674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.478617 12674 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:39543
I20260811 14:34:36.478659 12813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:39543 every 8 connection(s)
I20260811 14:34:36.479022 12674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:34:36.484297 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.479895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:40045 (local address 127.23.186.254:41343)
0811 14:34:36.479987 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.479990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.480174 (+ 184us) server_negotiation.cc:407] Connection header received
0811 14:34:36.480667 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.480668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.480679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.480795 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:36.481454 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.481961 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.482953 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.483139 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.483190 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.484088 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:36.484089 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:36.484089 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:36.484094 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:36.484106 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:36.484109 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:36.484148 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.484212 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.484234 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 14:34:36.484530 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.479847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41343 (local address 127.23.186.194:40045)
0811 14:34:36.480043 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.480080 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.480562 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.480842 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.480846 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.480855 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:36.481370 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.481374 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.482055 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.482056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.482874 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.482877 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.483215 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.483216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.483329 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.483941 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:36.483942 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:36.483973 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:36.483998 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:36.484149 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:36.484151 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:36.484153 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.484200 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.484280 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260811 14:34:36.484941 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 12674
I20260811 14:34:36.485080 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:34:36.485122 12814 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41343
I20260811 14:34:36.485225 12814 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:36.485332 24299 external_mini_cluster.cc:1428] Setting key 221a739ac7d624420c660d11f03a54c1
I20260811 14:34:36.485450 12814 heartbeater.cc:511] Master 127.23.186.254:41343 requested a full tablet report, sending...
I20260811 14:34:36.485859 12453 ts_manager.cc:194] Registered new tserver with Master: cc1a534daf1c4e349448eb6d025f7a65 (127.23.186.194:39543)
I20260811 14:34:36.486270 12453 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.194:40045
I20260811 14:34:36.486370 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41343
--builtin_ntp_servers=127.23.186.212:45517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:36.572134 12818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:36.572365 12818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:36.572389 12818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:36.573559 12818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:36.573602 12818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:36.573637 12818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:36.574999 12818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:45517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:41343
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:36.575232 12818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:36.575554 12818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:36.580955 12826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.580955 12823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:36.580977 12824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:36.581601 12818 server_base.cc:1034] running on GCE node
I20260811 14:34:36.581823 12818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:36.582114 12818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:36.583287 12818 hybrid_clock.cc:648] HybridClock initialized: now 1786458876583268 us; error 45 us; skew 500 ppm
I20260811 14:34:36.584823 12818 webserver.cc:460] Webserver started at http://127.23.186.195:42827/ using document root <none> and password file <none>
I20260811 14:34:36.585115 12818 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:36.585355 12818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:36.585549 12818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:36.586722 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "e22416a342b54ff0a090ddd938aa0703"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "1915009db7a20f1f68577aad18b2fe8d"
server_key_iv: "f3b8b866ac08da1cbed624cd52daa95c"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.587097 12818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "e22416a342b54ff0a090ddd938aa0703"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "1915009db7a20f1f68577aad18b2fe8d"
server_key_iv: "f3b8b866ac08da1cbed624cd52daa95c"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.588495 12818 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:36.589419 12832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.589634 12818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:36.589751 12818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "e22416a342b54ff0a090ddd938aa0703"
format_stamp: "Formatted at 2026-08-11 14:34:36 on dist-test-slave-cbsf"
server_key: "1915009db7a20f1f68577aad18b2fe8d"
server_key_iv: "f3b8b866ac08da1cbed624cd52daa95c"
server_key_version: "encryptionkey@0"
I20260811 14:34:36.589846 12818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:36.606933 12818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:36.607318 12818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:36.607491 12818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:36.607759 12818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:36.608126 12818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:36.608183 12818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.608227 12818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:36.608255 12818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:36.614600 12818 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:44755
I20260811 14:34:36.614660 12959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:44755 every 8 connection(s)
I20260811 14:34:36.615523 12818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:36.621464 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.616435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:58701 (local address 127.23.186.254:41343)
0811 14:34:36.616637 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.616640 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.616899 (+ 259us) server_negotiation.cc:407] Connection header received
0811 14:34:36.617489 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.617491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.617505 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.617555 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:36.618290 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.618815 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.619848 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.620069 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.620151 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.621147 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:36.621149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:36.621149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:36.621155 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:36.621172 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:36.621176 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:36.621231 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.621341 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.621376 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 14:34:36.621688 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.616407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:41343 (local address 127.23.186.195:58701)
0811 14:34:36.616759 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.616805 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.617371 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.617630 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.617635 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.617645 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:36.618188 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.618192 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.618921 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.618922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.619749 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.619751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.620160 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.620161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.620304 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.620981 (+ 677us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:36.620981 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:36.621015 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:36.621040 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:36.621241 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:36.621246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:36.621250 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.621331 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.621441 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":96,"threads_started":1}
I20260811 14:34:36.622243 12961 heartbeater.cc:347] Connected to a master server at 127.23.186.254:41343
I20260811 14:34:36.622354 12961 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:36.622566 12961 heartbeater.cc:511] Master 127.23.186.254:41343 requested a full tablet report, sending...
I20260811 14:34:36.622763 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 12818
I20260811 14:34:36.622872 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:34:36.623063 12453 ts_manager.cc:194] Registered new tserver with Master: e22416a342b54ff0a090ddd938aa0703 (127.23.186.195:44755)
I20260811 14:34:36.623123 24299 external_mini_cluster.cc:1428] Setting key 333f2ab79d882535427d50873298d4a7
I20260811 14:34:36.623490 12453 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.195:58701
I20260811 14:34:36.624131 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:34:36.628489 12502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.625598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34832 (local address 127.23.186.254:41343)
0811 14:34:36.625662 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.625664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.625754 (+ 90us) server_negotiation.cc:407] Connection header received
0811 14:34:36.625850 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.625850 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.625862 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.625908 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:36.626067 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.626538 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.627250 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.627456 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.627515 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.628290 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:36.628291 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:36.628292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:36.628296 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:36.628309 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:36.628313 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:36.628358 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.628393 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.628429 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 14:34:36.630240 12453 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:34:36.631893 12453 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:34:36.638316 12971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.634053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:41203 (local address 127.0.0.1:59500)
0811 14:34:36.634373 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.634421 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.634503 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.634894 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.634895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.634898 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:36.634960 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.634962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.635857 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.635859 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.637245 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.637249 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.638003 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.638005 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.638085 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.638099 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.638177 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.638234 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":24,"thread_start_us":79,"threads_started":1}
I20260811 14:34:36.638327 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.634139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59500 (local address 127.23.186.193:41203)
0811 14:34:36.634492 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.634496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.634513 (+ 17us) server_negotiation.cc:407] Connection header received
0811 14:34:36.634590 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.634592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.634610 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.634882 (+ 272us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:36.635028 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.635738 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.637355 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.637892 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.638003 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.638105 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.638192 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.638234 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":130,"threads_started":1}
I20260811 14:34:36.638474 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.634150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59008 (local address 127.23.186.194:39543)
0811 14:34:36.634382 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.634388 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.634589 (+ 201us) server_negotiation.cc:407] Connection header received
0811 14:34:36.634632 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.634634 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.634649 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.634717 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:36.634975 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.635721 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.637231 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.638002 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.638077 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.638155 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.638335 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.638374 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":114,"threads_started":1}
I20260811 14:34:36.638876 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.634059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:39543 (local address 127.0.0.1:59008)
0811 14:34:36.634389 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.634422 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.634499 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.634801 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.634804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.634807 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:36.634877 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.634879 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.635810 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.635812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.637137 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.637140 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.638190 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.638191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.638253 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.638257 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.638752 (+ 495us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.638800 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":70,"thread_start_us":67,"threads_started":1}
I20260811 14:34:36.639142 12595 tablet_service.cc:1467] Processing CreateTablet for tablet 60c33ab0f8df45e59bf7fea355332e53 (DEFAULT_TABLE table=test-table [id=0bf87f7fffdc4b18a1d41f2eda189354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:36.639593 12595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60c33ab0f8df45e59bf7fea355332e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:36.639775 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.634278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:44755 (local address 127.0.0.1:57182)
0811 14:34:36.634675 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.634686 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.634737 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.636070 (+ 1333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.636071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.636074 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:36.636112 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.636113 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.637378 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.637383 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.638414 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.638416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.639536 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.639537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.639605 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.639610 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.639662 (+ 52us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.639705 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":66,"threads_started":1}
I20260811 14:34:36.639667 12740 tablet_service.cc:1467] Processing CreateTablet for tablet 60c33ab0f8df45e59bf7fea355332e53 (DEFAULT_TABLE table=test-table [id=0bf87f7fffdc4b18a1d41f2eda189354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:36.640076 12740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60c33ab0f8df45e59bf7fea355332e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:36.640112 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.634480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.23.186.195:44755)
0811 14:34:36.635893 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.635897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.635907 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:36.635955 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.635957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.635973 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.636057 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:36.636182 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.637302 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.638520 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.639335 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.639407 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.639497 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.639965 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.640017 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1372,"thread_start_us":92,"threads_started":1}
I20260811 14:34:36.640686 12885 tablet_service.cc:1467] Processing CreateTablet for tablet 60c33ab0f8df45e59bf7fea355332e53 (DEFAULT_TABLE table=test-table [id=0bf87f7fffdc4b18a1d41f2eda189354]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:36.641078 12885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 60c33ab0f8df45e59bf7fea355332e53. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:36.642700 12976 tablet_bootstrap.cc:492] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f: Bootstrap starting.
I20260811 14:34:36.643378 12976 tablet_bootstrap.cc:654] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:36.643689 12977 tablet_bootstrap.cc:492] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Bootstrap starting.
I20260811 14:34:36.643752 12976 log.cc:824] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:36.644441 12977 tablet_bootstrap.cc:654] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:36.644518 12979 tablet_bootstrap.cc:492] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Bootstrap starting.
I20260811 14:34:36.644626 12976 tablet_bootstrap.cc:492] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f: No bootstrap required, opened a new log
I20260811 14:34:36.644737 12976 ts_tablet_manager.cc:1397] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:34:36.645033 12977 log.cc:824] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:36.645107 12979 tablet_bootstrap.cc:654] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:36.645334 12979 log.cc:824] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:36.645944 12977 tablet_bootstrap.cc:492] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: No bootstrap required, opened a new log
I20260811 14:34:36.646034 12977 ts_tablet_manager.cc:1397] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:34:36.646167 12979 tablet_bootstrap.cc:492] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: No bootstrap required, opened a new log
I20260811 14:34:36.646236 12979 ts_tablet_manager.cc:1397] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:34:36.647102 12976 raft_consensus.cc:348] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.647269 12976 raft_consensus.cc:374] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:36.647315 12976 raft_consensus.cc:729] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60911084a979477fbc5eca3a9c368f7f, State: Initialized, Role: FOLLOWER
I20260811 14:34:36.647461 12976 consensus_queue.cc:260] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.647746 12976 ts_tablet_manager.cc:1428] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:36.647878 12671 heartbeater.cc:503] Master 127.23.186.254:41343 was elected leader, sending a full tablet report...
I20260811 14:34:36.648761 12977 raft_consensus.cc:348] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.648803 12979 raft_consensus.cc:348] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.648936 12979 raft_consensus.cc:374] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:36.648936 12977 raft_consensus.cc:374] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:36.648981 12979 raft_consensus.cc:729] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e22416a342b54ff0a090ddd938aa0703, State: Initialized, Role: FOLLOWER
I20260811 14:34:36.648983 12977 raft_consensus.cc:729] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1a534daf1c4e349448eb6d025f7a65, State: Initialized, Role: FOLLOWER
I20260811 14:34:36.649121 12979 consensus_queue.cc:260] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.649189 12977 consensus_queue.cc:260] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.649403 12979 ts_tablet_manager.cc:1428] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:36.649422 12977 ts_tablet_manager.cc:1428] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:34:36.649513 12814 heartbeater.cc:503] Master 127.23.186.254:41343 was elected leader, sending a full tablet report...
I20260811 14:34:36.649528 12961 heartbeater.cc:503] Master 127.23.186.254:41343 was elected leader, sending a full tablet report...
I20260811 14:34:36.663424 12984 raft_consensus.cc:482] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:36.663599 12984 raft_consensus.cc:504] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.664044 12984 leader_election.cc:288] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 60911084a979477fbc5eca3a9c368f7f (127.23.186.193:41203), e22416a342b54ff0a090ddd938aa0703 (127.23.186.195:44755)
I20260811 14:34:36.667999 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.664310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:39965 (local address 127.23.186.195:44755)
0811 14:34:36.664355 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.664357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.664553 (+ 196us) server_negotiation.cc:407] Connection header received
0811 14:34:36.664580 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.664580 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.664587 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.664653 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:36.664829 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.665532 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.667049 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.667610 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.667694 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.667704 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.667872 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.667914 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:36.667989 12973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.664298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:43019 (local address 127.23.186.193:41203)
0811 14:34:36.664346 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.664349 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.664359 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:36.664390 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.664390 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.664401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.664465 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:36.664648 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.665499 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.667050 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.667624 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.667694 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.667704 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.667872 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.667915 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:34:36.668004 12816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.664153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:41203 (local address 127.23.186.194:43019)
0811 14:34:36.664222 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.664237 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.664286 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.664489 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.664490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.664492 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:36.664568 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.664569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.665596 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.665597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.666953 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.666956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.667707 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.667707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.667788 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.667792 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.667852 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.667920 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":73}
I20260811 14:34:36.668011 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.664241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:44755 (local address 127.23.186.194:39965)
0811 14:34:36.664410 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.664461 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:36.664513 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:36.664668 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:36.664668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:36.664670 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:36.664755 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.664756 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.665600 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.665601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.666953 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:36.666956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.667708 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.667709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:36.667789 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.667792 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:36.667852 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 14:34:36.667908 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":64,"thread_start_us":125,"threads_started":1}
I20260811 14:34:36.668501 12619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c33ab0f8df45e59bf7fea355332e53" candidate_uuid: "cc1a534daf1c4e349448eb6d025f7a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60911084a979477fbc5eca3a9c368f7f" is_pre_election: true
I20260811 14:34:36.668745 12619 raft_consensus.cc:2454] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1a534daf1c4e349448eb6d025f7a65 in term 0.
I20260811 14:34:36.668726 12907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c33ab0f8df45e59bf7fea355332e53" candidate_uuid: "cc1a534daf1c4e349448eb6d025f7a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e22416a342b54ff0a090ddd938aa0703" is_pre_election: true
I20260811 14:34:36.668875 12907 raft_consensus.cc:2454] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1a534daf1c4e349448eb6d025f7a65 in term 0.
I20260811 14:34:36.669011 12692 leader_election.cc:302] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60911084a979477fbc5eca3a9c368f7f, cc1a534daf1c4e349448eb6d025f7a65; no voters:
I20260811 14:34:36.669188 12984 raft_consensus.cc:2781] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:34:36.669238 12984 raft_consensus.cc:482] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:34:36.669277 12984 raft_consensus.cc:3037] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:36.670126 12984 raft_consensus.cc:504] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.670284 12984 leader_election.cc:288] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [CANDIDATE]: Term 1 election: Requested vote from peers 60911084a979477fbc5eca3a9c368f7f (127.23.186.193:41203), e22416a342b54ff0a090ddd938aa0703 (127.23.186.195:44755)
I20260811 14:34:36.670430 12619 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c33ab0f8df45e59bf7fea355332e53" candidate_uuid: "cc1a534daf1c4e349448eb6d025f7a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60911084a979477fbc5eca3a9c368f7f"
I20260811 14:34:36.670506 12907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "60c33ab0f8df45e59bf7fea355332e53" candidate_uuid: "cc1a534daf1c4e349448eb6d025f7a65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e22416a342b54ff0a090ddd938aa0703"
I20260811 14:34:36.670540 12619 raft_consensus.cc:3037] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:36.670607 12907 raft_consensus.cc:3037] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:36.671280 12619 raft_consensus.cc:2454] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1a534daf1c4e349448eb6d025f7a65 in term 1.
I20260811 14:34:36.671423 12907 raft_consensus.cc:2454] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1a534daf1c4e349448eb6d025f7a65 in term 1.
I20260811 14:34:36.671511 12692 leader_election.cc:302] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60911084a979477fbc5eca3a9c368f7f, cc1a534daf1c4e349448eb6d025f7a65; no voters:
I20260811 14:34:36.671602 12984 raft_consensus.cc:2781] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:36.671783 12984 raft_consensus.cc:686] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 1 LEADER]: Becoming Leader. State: Replica: cc1a534daf1c4e349448eb6d025f7a65, State: Running, Role: LEADER
I20260811 14:34:36.671895 12984 consensus_queue.cc:237] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } }
I20260811 14:34:36.672832 12453 catalog_manager.cc:5495] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1a534daf1c4e349448eb6d025f7a65 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "cc1a534daf1c4e349448eb6d025f7a65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1a534daf1c4e349448eb6d025f7a65" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 39543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 } health_report { overall_health: UNKNOWN } } }
I20260811 14:34:36.680889 12972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.678667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59010 (local address 127.23.186.194:39543)
0811 14:34:36.678728 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:36.678730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:36.678739 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:34:36.678797 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:36.678798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:36.678814 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:36.678873 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:34:36.678972 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.679435 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.680160 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:36.680340 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:36.680390 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:36.680523 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:34:36.680626 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:34:36.680733 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:36.680791 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 14:34:36.680809 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:34:36.682834 12619 raft_consensus.cc:1261] T 60c33ab0f8df45e59bf7fea355332e53 P 60911084a979477fbc5eca3a9c368f7f [term 1 FOLLOWER]: Refusing update from remote peer cc1a534daf1c4e349448eb6d025f7a65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:36.683193 12907 raft_consensus.cc:1261] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 1 FOLLOWER]: Refusing update from remote peer cc1a534daf1c4e349448eb6d025f7a65: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:36.683203 12984 consensus_queue.cc:1035] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60911084a979477fbc5eca3a9c368f7f" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 41203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:36.683486 12984 consensus_queue.cc:1035] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e22416a342b54ff0a090ddd938aa0703" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 44755 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:36.688738 12454 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:36.688860 12454 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:36.689770 12454 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c99e9bda98449eaad327ed16ee2c160: Sending DeleteTablet for 3 replicas of tablet 60c33ab0f8df45e59bf7fea355332e53
I20260811 14:34:36.690086 12740 tablet_service.cc:1514] Processing DeleteTablet for tablet 60c33ab0f8df45e59bf7fea355332e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:36 UTC) from {username='slave'} at 127.0.0.1:59008
I20260811 14:34:36.690140 12885 tablet_service.cc:1514] Processing DeleteTablet for tablet 60c33ab0f8df45e59bf7fea355332e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:36 UTC) from {username='slave'} at 127.0.0.1:57182
I20260811 14:34:36.690193 12595 tablet_service.cc:1514] Processing DeleteTablet for tablet 60c33ab0f8df45e59bf7fea355332e53 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:36 UTC) from {username='slave'} at 127.0.0.1:59500
I20260811 14:34:36.690340 13001 tablet_replica.cc:330] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: stopping tablet replica
I20260811 14:34:36.690340 13002 tablet_replica.cc:330] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: stopping tablet replica
I20260811 14:34:36.690443 13002 raft_consensus.cc:2229] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:36.690444 13001 raft_consensus.cc:2229] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:34:36.690547 13001 raft_consensus.cc:2258] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:36.690552 13002 raft_consensus.cc:2258] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:36.690769 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 12522
I20260811 14:34:36.691009 13002 ts_tablet_manager.cc:1905] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:36.691009 13001 ts_tablet_manager.cc:1905] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:36.692860 13002 ts_tablet_manager.cc:1918] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:36.693015 13002 log.cc:1196] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/60c33ab0f8df45e59bf7fea355332e53
I20260811 14:34:36.693050 13001 ts_tablet_manager.cc:1918] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:36.693131 13001 log.cc:1196] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/60c33ab0f8df45e59bf7fea355332e53
I20260811 14:34:36.693347 13002 ts_tablet_manager.cc:1939] T 60c33ab0f8df45e59bf7fea355332e53 P e22416a342b54ff0a090ddd938aa0703: Deleting consensus metadata
I20260811 14:34:36.693421 13001 ts_tablet_manager.cc:1939] T 60c33ab0f8df45e59bf7fea355332e53 P cc1a534daf1c4e349448eb6d025f7a65: Deleting consensus metadata
I20260811 14:34:36.693847 12436 catalog_manager.cc:4841] TS e22416a342b54ff0a090ddd938aa0703 (127.23.186.195:44755): tablet 60c33ab0f8df45e59bf7fea355332e53 (table test-table [id=0bf87f7fffdc4b18a1d41f2eda189354]) successfully deleted
I20260811 14:34:36.693993 12436 catalog_manager.cc:4841] TS cc1a534daf1c4e349448eb6d025f7a65 (127.23.186.194:39543): tablet 60c33ab0f8df45e59bf7fea355332e53 (table test-table [id=0bf87f7fffdc4b18a1d41f2eda189354]) successfully deleted
W20260811 14:34:36.701568 12438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:41203 (error 108)
I20260811 14:34:36.701831 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 12674
I20260811 14:34:36.705456 12970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:36.705256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:41203 (local address 127.0.0.1:59508)
0811 14:34:36.705330 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:36.705398 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:41203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 14:34:36.705646 12438 catalog_manager.cc:4584] TS 60911084a979477fbc5eca3a9c368f7f (127.23.186.193:41203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 60c33ab0f8df45e59bf7fea355332e53: Network error: Client connection negotiation failed: client connection to 127.23.186.193:41203: connect: Connection refused (error 111)
I20260811 14:34:36.709590 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 12818
I20260811 14:34:36.716990 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 12419
2026-08-11T14:34:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (725 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:36 dist-test-slave-cbsf krb5kdc[13008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:36 dist-test-slave-cbsf krb5kdc[13008](info): set up 2 sockets
Aug 11 14:34:36 dist-test-slave-cbsf krb5kdc[13008](info): commencing operation
krb5kdc: starting...
W20260811 14:34:37.850835 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:37 dist-test-slave-cbsf krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:34:37.981370 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:43583
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:43583
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:38.067394 13022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:38.067649 13022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:38.067687 13022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:38.068873 13022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:38.068956 13022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:38.068984 13022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:38.069015 13022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:38.069037 13022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:38.070520 13022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:43583
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:43583
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:38.070802 13022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:38.071038 13022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:38.075975 13028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.075990 13031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.075994 13027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:38.076213 13022 server_base.cc:1034] running on GCE node
I20260811 14:34:38.076483 13022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:38.076823 13022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:38.077972 13022 hybrid_clock.cc:648] HybridClock initialized: now 1786458878077960 us; error 27 us; skew 500 ppm
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:38.080535 13022 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:34:38.081030 13022 webserver.cc:460] Webserver started at http://127.23.186.254:41975/ using document root <none> and password file <none>
I20260811 14:34:38.081236 13022 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:38.081290 13022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:38.081385 13022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:38.082460 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "3cbf8eb014d341538e9654fa0a048229"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "6019497df24da9904317511d408534d1"
server_key_iv: "1e2f6f86a3cae42cc1a52d2e8e90b400"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.082769 13022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "3cbf8eb014d341538e9654fa0a048229"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "6019497df24da9904317511d408534d1"
server_key_iv: "1e2f6f86a3cae42cc1a52d2e8e90b400"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.084121 13022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:38.084949 13038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.085129 13022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:38.085240 13022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "3cbf8eb014d341538e9654fa0a048229"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "6019497df24da9904317511d408534d1"
server_key_iv: "1e2f6f86a3cae42cc1a52d2e8e90b400"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.085318 13022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:38.103376 13022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:38.105494 13022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:38.105676 13022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:38.109907 13022 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:43583
I20260811 14:34:38.109947 13101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:43583 every 8 connection(s)
I20260811 14:34:38.110356 13022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:38.111320 13102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:38.113806 13102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229: Bootstrap starting.
I20260811 14:34:38.114444 13102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:38.114756 13102 log.cc:824] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:38.115568 13102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229: No bootstrap required, opened a new log
I20260811 14:34:38.117270 13102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbf8eb014d341538e9654fa0a048229" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43583 } }
I20260811 14:34:38.117420 13102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:38.117443 13102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cbf8eb014d341538e9654fa0a048229, State: Initialized, Role: FOLLOWER
I20260811 14:34:38.117627 13102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbf8eb014d341538e9654fa0a048229" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43583 } }
I20260811 14:34:38.117717 13102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:38.117754 13102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:38.117825 13102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:38.117820 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13022
I20260811 14:34:38.117969 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:38.118191 24299 external_mini_cluster.cc:1428] Setting key 4a336357d86783ba693d7b376aaf1efb
I20260811 14:34:38.118548 13102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbf8eb014d341538e9654fa0a048229" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43583 } }
I20260811 14:34:38.118671 13102 leader_election.cc:302] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cbf8eb014d341538e9654fa0a048229; no voters:
I20260811 14:34:38.118911 13102 leader_election.cc:288] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:38.118968 13109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:38.119119 13109 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [term 1 LEADER]: Becoming Leader. State: Replica: 3cbf8eb014d341538e9654fa0a048229, State: Running, Role: LEADER
I20260811 14:34:38.119266 13102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:38.119279 13109 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbf8eb014d341538e9654fa0a048229" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43583 } }
I20260811 14:34:38.120002 13111 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cbf8eb014d341538e9654fa0a048229. Latest consensus state: current_term: 1 leader_uuid: "3cbf8eb014d341538e9654fa0a048229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbf8eb014d341538e9654fa0a048229" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43583 } } }
I20260811 14:34:38.120090 13111 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:38.120477 13118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:38.120513 13110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cbf8eb014d341538e9654fa0a048229" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cbf8eb014d341538e9654fa0a048229" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43583 } } }
I20260811 14:34:38.120837 13110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:38.121089 13118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:38.122670 13118 catalog_manager.cc:1347] Generated new cluster ID: 096fd1d935e8442195d6b2eb5c55d861
I20260811 14:34:38.122723 13118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:38.130601 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.118743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.23.186.254:43583)
0811 14:34:38.118983 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.118993 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.119030 (+ 37us) server_negotiation.cc:407] Connection header received
0811 14:34:38.119176 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.119183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.119246 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.120274 (+ 1028us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:38.120408 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.121856 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.122931 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.123536 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.123636 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.126640 (+ 3004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:38.126645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:38.126645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:38.126659 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:38.128876 (+ 2217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.129264 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.129268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.129268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.129409 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.129612 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.129613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.129613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.129951 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:38.130139 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.130218 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.130371 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":109,"threads_started":1}
I20260811 14:34:38.137182 13118 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:38.137708 13118 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:38.143615 13118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229: Generated new TSK 0
I20260811 14:34:38.143841 13118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:34:38.180032 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43583
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:38.265516 13131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:38.265774 13131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:38.265818 13131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:38.267036 13131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:38.267118 13131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:38.267165 13131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:38.268754 13131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:38.269032 13131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:38.269284 13131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:38.274389 13140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.274420 13137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.274392 13138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:38.274693 13131 server_base.cc:1034] running on GCE node
I20260811 14:34:38.274919 13131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:38.275173 13131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:38.276335 13131 hybrid_clock.cc:648] HybridClock initialized: now 1786458878276328 us; error 36 us; skew 500 ppm
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:38.279052 13131 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:34:38.279521 13131 webserver.cc:460] Webserver started at http://127.23.186.193:34621/ using document root <none> and password file <none>
I20260811 14:34:38.279716 13131 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:38.279759 13131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:38.279851 13131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:38.280972 13131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "c52d0dea3e23f50dccca8e94e20c3e6b"
server_key_iv: "5d078b560faa870165fc9c4bebb76417"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.281297 13131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "c52d0dea3e23f50dccca8e94e20c3e6b"
server_key_iv: "5d078b560faa870165fc9c4bebb76417"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.282604 13131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:38.283313 13149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.283501 13131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:38.283591 13131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "c52d0dea3e23f50dccca8e94e20c3e6b"
server_key_iv: "5d078b560faa870165fc9c4bebb76417"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.283653 13131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:38.303803 13131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:38.305902 13131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:38.306023 13131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:38.306242 13131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:38.306519 13131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:38.306551 13131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.306576 13131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:38.306588 13131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.312456 13131 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:36155
I20260811 14:34:38.312731 13277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:36155 every 8 connection(s)
I20260811 14:34:38.312930 13131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:38.316824 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13131
I20260811 14:34:38.316962 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:34:38.317201 24299 external_mini_cluster.cc:1428] Setting key ef0727c01409df27e6e0a4bec8261441
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:38.323746 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.313763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:37029 (local address 127.23.186.254:43583)
0811 14:34:38.313843 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.313846 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.314033 (+ 187us) server_negotiation.cc:407] Connection header received
0811 14:34:38.314592 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.314594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.314620 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.314678 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:38.315688 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.316218 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.317548 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.317838 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.317911 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.320511 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:38.320513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:38.320513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:38.320523 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:38.322221 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.322629 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.322630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.322631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.322706 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.322946 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.322948 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.322948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.323126 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:38.323209 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.323496 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.323599 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:34:38.323760 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.313723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:43583 (local address 127.23.186.193:37029)
0811 14:34:38.313914 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.313951 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.314485 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.314749 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.314753 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.315379 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:38.315580 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.315585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.316324 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.316325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.317459 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.317463 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.317934 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.317935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.318083 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.318667 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:38.318676 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:38.320385 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:38.322304 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:38.322311 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:38.322313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:38.322543 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:38.322779 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:38.322780 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:38.322780 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:38.322880 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:38.323202 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:38.323208 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:38.323357 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.323445 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.323572 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":91,"threads_started":1}
I20260811 14:34:38.324434 13278 heartbeater.cc:347] Connected to a master server at 127.23.186.254:43583
I20260811 14:34:38.324554 13278 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:38.324800 13278 heartbeater.cc:511] Master 127.23.186.254:43583 requested a full tablet report, sending...
I20260811 14:34:38.325376 13059 ts_manager.cc:194] Registered new tserver with Master: b5212a9e0df64b8fb4a9d3b8d06b3bf4 (127.23.186.193:36155)
I20260811 14:34:38.325987 13059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:37029
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:34:38.345629 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43583
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:38.430858 13284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:38.431078 13284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:38.431104 13284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:38.432279 13284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:38.432333 13284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:38.432361 13284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:38.433861 13284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:38.434129 13284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:38.434381 13284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:38.439575 13291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.439560 13292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:38.439671 13284 server_base.cc:1034] running on GCE node
W20260811 14:34:38.439560 13294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:38.439946 13284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:38.440212 13284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:38.441352 13284 hybrid_clock.cc:648] HybridClock initialized: now 1786458878441325 us; error 42 us; skew 500 ppm
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:38.443789 13284 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:34:38.444192 13284 webserver.cc:460] Webserver started at http://127.23.186.194:35651/ using document root <none> and password file <none>
I20260811 14:34:38.444380 13284 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:38.444422 13284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:38.444517 13284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:38.445608 13284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "2e7d178fc5c39c56520da928d88585e0"
server_key_iv: "65e1bd94e285b3f941c14ee20eb44db2"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.445920 13284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "2e7d178fc5c39c56520da928d88585e0"
server_key_iv: "65e1bd94e285b3f941c14ee20eb44db2"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.447214 13284 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:34:38.447917 13303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.448072 13284 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:38.448171 13284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "2e7d178fc5c39c56520da928d88585e0"
server_key_iv: "65e1bd94e285b3f941c14ee20eb44db2"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.448232 13284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:38.463781 13284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:38.466200 13284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:38.466337 13284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:38.466568 13284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:38.466856 13284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:38.466888 13284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.466910 13284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:38.466923 13284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.473135 13284 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:45549
I20260811 14:34:38.473222 13431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:45549 every 8 connection(s)
I20260811 14:34:38.473562 13284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:38.482116 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13284
I20260811 14:34:38.482260 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:34:38.482478 24299 external_mini_cluster.cc:1428] Setting key 04573da5efe9b67c78278302f2afafca
I20260811 14:34:38.482867 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.474360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:60751 (local address 127.23.186.254:43583)
0811 14:34:38.474438 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.474440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.474656 (+ 216us) server_negotiation.cc:407] Connection header received
0811 14:34:38.475135 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.475136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.475146 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.475210 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:38.476182 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.476681 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.477744 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.477946 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.478003 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.480100 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:38.480102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:38.480102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:38.480115 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:38.481398 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.481787 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.481788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.481789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.481841 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.482143 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.482144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.482144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.482302 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:38.482371 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.482683 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.482772 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 14:34:38.483088 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.474319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:43583 (local address 127.23.186.194:60751)
0811 14:34:38.474520 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.474565 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.475047 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.475263 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.475267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.475883 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:38.476097 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.476102 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.476808 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.476809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.477656 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.477659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.478019 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.478020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.478119 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.478607 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:38.478616 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:38.480014 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:38.481497 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:38.481505 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:38.481507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:38.481724 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:38.481917 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:38.481918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:38.481918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:38.482074 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:38.482365 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:38.482374 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:38.482512 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.482658 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.482837 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
I20260811 14:34:38.483592 13432 heartbeater.cc:347] Connected to a master server at 127.23.186.254:43583
I20260811 14:34:38.483708 13432 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:38.483904 13432 heartbeater.cc:511] Master 127.23.186.254:43583 requested a full tablet report, sending...
I20260811 14:34:38.484295 13059 ts_manager.cc:194] Registered new tserver with Master: 4dbe50bc0ee34a348eb0be8a93a52ad6 (127.23.186.194:45549)
I20260811 14:34:38.484679 13059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:60751
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:34:38.509186 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43583
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:41035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:38.594412 13437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:38.594633 13437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:38.594658 13437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:38.595842 13437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:38.595888 13437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:38.595942 13437 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:38.597488 13437 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:41035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:43583
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:38.597759 13437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:38.598002 13437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:38.603143 13444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.603159 13443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:38.603122 13446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:38.603361 13437 server_base.cc:1034] running on GCE node
I20260811 14:34:38.603565 13437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:38.603770 13437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:38.604929 13437 hybrid_clock.cc:648] HybridClock initialized: now 1786458878604916 us; error 28 us; skew 500 ppm
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:38.607733 13437 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:34:38.608237 13437 webserver.cc:460] Webserver started at http://127.23.186.195:43085/ using document root <none> and password file <none>
I20260811 14:34:38.608495 13437 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:38.608543 13437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:38.608759 13437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:38.609865 13437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "650b258cc1b74350876a6c305b230d59"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "29b53d217ac7994b19cde89a82b61516"
server_key_iv: "c2fc0ce43ac1b18de8f5d34468407b25"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.610234 13437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "650b258cc1b74350876a6c305b230d59"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "29b53d217ac7994b19cde89a82b61516"
server_key_iv: "c2fc0ce43ac1b18de8f5d34468407b25"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.611613 13437 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:38.612299 13453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.612478 13437 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:38.612586 13437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "650b258cc1b74350876a6c305b230d59"
format_stamp: "Formatted at 2026-08-11 14:34:38 on dist-test-slave-cbsf"
server_key: "29b53d217ac7994b19cde89a82b61516"
server_key_iv: "c2fc0ce43ac1b18de8f5d34468407b25"
server_key_version: "encryptionkey@0"
I20260811 14:34:38.612663 13437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:38.624428 13437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:38.626891 13437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:38.627064 13437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:38.627319 13437 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:38.627662 13437 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:38.627717 13437 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.627758 13437 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:38.627789 13437 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:38.634007 13437 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:40355
I20260811 14:34:38.634057 13583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:40355 every 8 connection(s)
I20260811 14:34:38.634403 13437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:38.635360 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13437
I20260811 14:34:38.635470 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:34:38.635660 24299 external_mini_cluster.cc:1428] Setting key 039f170b50edb36133e7c2b0a89c3f3c
Aug 11 14:34:38 dist-test-slave-cbsf krb5kdc[13008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458878, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:38.644029 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.635189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:43091 (local address 127.23.186.254:43583)
0811 14:34:38.635260 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.635263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.635461 (+ 198us) server_negotiation.cc:407] Connection header received
0811 14:34:38.636088 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.636089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.636102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.636148 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:38.637271 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.637779 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.638764 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.638945 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.638995 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.641402 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:38.641404 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:38.641404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:38.641412 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:38.642644 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.643021 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.643022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.643022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.643076 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.643320 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.643321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.643321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.643474 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:38.643535 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.643893 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.643961 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 14:34:38.644361 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.635132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:43583 (local address 127.23.186.195:43091)
0811 14:34:38.635322 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.635367 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.635991 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.636224 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.636229 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.636972 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:38.637193 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.637197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.637858 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.637859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.638670 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.638672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.639015 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.639015 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.639131 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.639619 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:38.639628 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:38.641309 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:38.642719 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:38.642726 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:38.642728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:38.642957 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:38.643136 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:38.643136 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:38.643137 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:38.643236 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:38.643571 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:38.643580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:38.643761 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.643863 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.644004 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":115,"threads_started":1}
I20260811 14:34:38.644981 13585 heartbeater.cc:347] Connected to a master server at 127.23.186.254:43583
I20260811 14:34:38.645102 13585 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:38.645308 13585 heartbeater.cc:511] Master 127.23.186.254:43583 requested a full tablet report, sending...
I20260811 14:34:38.645716 13059 ts_manager.cc:194] Registered new tserver with Master: 650b258cc1b74350876a6c305b230d59 (127.23.186.195:40355)
I20260811 14:34:38.646091 13059 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:43091
I20260811 14:34:38.647428 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:34:38.655575 13107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.648797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.23.186.254:43583)
0811 14:34:38.648867 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.648869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.648980 (+ 111us) server_negotiation.cc:407] Connection header received
0811 14:34:38.649069 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.649070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.649085 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.649142 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:38.649716 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.650197 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.651393 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.651609 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.651685 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.653181 (+ 1496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:38.653183 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:38.653183 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:38.653191 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:38.654322 (+ 1131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.654725 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.654726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.654726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.654788 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:38.655005 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:38.655006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:38.655006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:38.655154 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:38.655268 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.655395 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.655460 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:34:38.657243 13059 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40050:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:34:38.658933 13059 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:34:38.665982 13597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.661403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:40355 (local address 127.0.0.1:47720)
0811 14:34:38.661854 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.661865 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.661917 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.662196 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.662200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.662205 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:38.662304 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.662307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.663658 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.663658 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.664805 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.664808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.665696 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.665697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.665765 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.665769 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.665838 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.665885 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":23,"thread_start_us":95,"threads_started":1}
I20260811 14:34:38.665983 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.661662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47720 (local address 127.23.186.195:40355)
0811 14:34:38.661957 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.661961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.661971 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:38.662007 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.662009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.662024 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.662096 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:38.662396 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.663549 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.664898 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.665626 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.665696 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.665746 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.665855 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.665895 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":103,"threads_started":1}
I20260811 14:34:38.666196 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.661684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.23.186.193:36155)
0811 14:34:38.661860 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.661864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.662114 (+ 250us) server_negotiation.cc:407] Connection header received
0811 14:34:38.662164 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.662167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.662183 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.662257 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:38.662480 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.663191 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.664684 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.665475 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.665553 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.665618 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.665951 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.665999 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260811 14:34:38.666358 13598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.661405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:36155 (local address 127.0.0.1:36786)
0811 14:34:38.661974 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.661981 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.662037 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.662349 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.662350 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.662352 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:38.662399 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.662401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.663293 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.663294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.664556 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.664560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.665583 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.665584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.665666 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.665682 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.665752 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.665806 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":92,"threads_started":1}
I20260811 14:34:38.667135 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.661394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:45549 (local address 127.0.0.1:48478)
0811 14:34:38.661699 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.661740 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.661843 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.662197 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.662200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.662203 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:38.662304 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.662307 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.664896 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.664897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.665868 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.665870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.666907 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.666908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.666955 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.666958 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.667027 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.667068 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":2,"thread_start_us":74,"threads_started":1}
I20260811 14:34:38.667160 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.661508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.23.186.194:45549)
0811 14:34:38.661677 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.661683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.661948 (+ 265us) server_negotiation.cc:407] Connection header received
0811 14:34:38.661999 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.662001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.662020 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.662093 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:38.662396 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.663545 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.665965 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.666748 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.666815 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.666868 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.667042 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.667082 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260811 14:34:38.667156 13206 tablet_service.cc:1467] Processing CreateTablet for tablet bd871ae5d41e4599968f6c630137fa5a (DEFAULT_TABLE table=test-table [id=ad2e6b2c24af4fcd8241c2b0fb0f52d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:38.667143 13508 tablet_service.cc:1467] Processing CreateTablet for tablet bd871ae5d41e4599968f6c630137fa5a (DEFAULT_TABLE table=test-table [id=ad2e6b2c24af4fcd8241c2b0fb0f52d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:38.667547 13508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd871ae5d41e4599968f6c630137fa5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:38.667547 13206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd871ae5d41e4599968f6c630137fa5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:38.671240 13602 tablet_bootstrap.cc:492] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Bootstrap starting.
I20260811 14:34:38.671833 13357 tablet_service.cc:1467] Processing CreateTablet for tablet bd871ae5d41e4599968f6c630137fa5a (DEFAULT_TABLE table=test-table [id=ad2e6b2c24af4fcd8241c2b0fb0f52d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:38.671972 13602 tablet_bootstrap.cc:654] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:38.672205 13357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd871ae5d41e4599968f6c630137fa5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:38.672561 13602 log.cc:824] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:38.672992 13603 tablet_bootstrap.cc:492] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: Bootstrap starting.
I20260811 14:34:38.673732 13602 tablet_bootstrap.cc:492] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: No bootstrap required, opened a new log
I20260811 14:34:38.673740 13603 tablet_bootstrap.cc:654] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:38.673815 13602 ts_tablet_manager.cc:1397] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:34:38.674338 13603 log.cc:824] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:38.675268 13603 tablet_bootstrap.cc:492] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: No bootstrap required, opened a new log
I20260811 14:34:38.675347 13603 ts_tablet_manager.cc:1397] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:34:38.675653 13608 tablet_bootstrap.cc:492] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Bootstrap starting.
I20260811 14:34:38.676012 13602 raft_consensus.cc:348] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.676208 13602 raft_consensus.cc:374] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:38.676245 13602 raft_consensus.cc:729] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 650b258cc1b74350876a6c305b230d59, State: Initialized, Role: FOLLOWER
I20260811 14:34:38.676312 13608 tablet_bootstrap.cc:654] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:38.676373 13602 consensus_queue.cc:260] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.676662 13602 ts_tablet_manager.cc:1428] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:38.676860 13585 heartbeater.cc:503] Master 127.23.186.254:43583 was elected leader, sending a full tablet report...
I20260811 14:34:38.677304 13608 log.cc:824] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:38.677744 13603 raft_consensus.cc:348] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.677938 13603 raft_consensus.cc:374] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:38.678000 13603 raft_consensus.cc:729] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5212a9e0df64b8fb4a9d3b8d06b3bf4, State: Initialized, Role: FOLLOWER
I20260811 14:34:38.678131 13603 consensus_queue.cc:260] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.678433 13603 ts_tablet_manager.cc:1428] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:38.678459 13278 heartbeater.cc:503] Master 127.23.186.254:43583 was elected leader, sending a full tablet report...
I20260811 14:34:38.679037 13608 tablet_bootstrap.cc:492] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: No bootstrap required, opened a new log
I20260811 14:34:38.679097 13608 ts_tablet_manager.cc:1397] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 14:34:38.680536 13608 raft_consensus.cc:348] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.680675 13608 raft_consensus.cc:374] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:38.680759 13608 raft_consensus.cc:729] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbe50bc0ee34a348eb0be8a93a52ad6, State: Initialized, Role: FOLLOWER
I20260811 14:34:38.680908 13608 consensus_queue.cc:260] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.681216 13608 ts_tablet_manager.cc:1428] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:34:38.681246 13432 heartbeater.cc:503] Master 127.23.186.254:43583 was elected leader, sending a full tablet report...
W20260811 14:34:38.724268 13433 tablet.cc:2369] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 14:34:38.813761 13279 tablet.cc:2369] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:34:38.826968 13611 raft_consensus.cc:482] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:38.827122 13611 raft_consensus.cc:504] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.827682 13611 leader_election.cc:288] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dbe50bc0ee34a348eb0be8a93a52ad6 (127.23.186.194:45549), 650b258cc1b74350876a6c305b230d59 (127.23.186.195:40355)
I20260811 14:34:38.830883 13599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.827972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:44785 (local address 127.23.186.194:45549)
0811 14:34:38.828027 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.828029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.828128 (+ 99us) server_negotiation.cc:407] Connection header received
0811 14:34:38.828217 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.828218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.828227 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.828293 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:38.828481 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.829027 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.830116 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.830508 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.830591 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.830609 (+ 18us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.830759 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.830802 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:34:38.830883 13600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.828013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:57937 (local address 127.23.186.195:40355)
0811 14:34:38.828067 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.828069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.828175 (+ 106us) server_negotiation.cc:407] Connection header received
0811 14:34:38.828265 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.828267 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.828277 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.828345 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:38.828524 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.829062 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.830112 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.830509 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.830593 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.830603 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.830752 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.830799 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:38.830921 13614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.827913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:40355 (local address 127.23.186.193:57937)
0811 14:34:38.828084 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.828095 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.828191 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.828351 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.828352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.828355 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:38.828452 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.828454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.829165 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.829166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.830009 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.830012 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.830593 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.830595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.830676 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.830680 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.830745 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.830805 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":5,"thread_start_us":83,"threads_started":1}
I20260811 14:34:38.830920 13613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.827889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:45549 (local address 127.23.186.193:44785)
0811 14:34:38.828034 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.828045 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:38.828141 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:38.828299 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:38.828300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:38.828303 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:38.828384 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.828391 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.829119 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.829120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.830011 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:38.830014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.830599 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.830600 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:38.830677 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.830681 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:38.830751 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:34:38.830800 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":49,"thread_start_us":63,"threads_started":1}
I20260811 14:34:38.831365 13380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd871ae5d41e4599968f6c630137fa5a" candidate_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" is_pre_election: true
I20260811 14:34:38.831367 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd871ae5d41e4599968f6c630137fa5a" candidate_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650b258cc1b74350876a6c305b230d59" is_pre_election: true
I20260811 14:34:38.831530 13530 raft_consensus.cc:2454] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5212a9e0df64b8fb4a9d3b8d06b3bf4 in term 0.
I20260811 14:34:38.831542 13380 raft_consensus.cc:2454] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5212a9e0df64b8fb4a9d3b8d06b3bf4 in term 0.
I20260811 14:34:38.831796 13150 leader_election.cc:302] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 650b258cc1b74350876a6c305b230d59, b5212a9e0df64b8fb4a9d3b8d06b3bf4; no voters:
I20260811 14:34:38.832003 13611 raft_consensus.cc:2781] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:34:38.832043 13611 raft_consensus.cc:482] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:34:38.832082 13611 raft_consensus.cc:3037] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:38.832907 13611 raft_consensus.cc:504] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.833069 13611 leader_election.cc:288] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [CANDIDATE]: Term 1 election: Requested vote from peers 4dbe50bc0ee34a348eb0be8a93a52ad6 (127.23.186.194:45549), 650b258cc1b74350876a6c305b230d59 (127.23.186.195:40355)
I20260811 14:34:38.833245 13530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd871ae5d41e4599968f6c630137fa5a" candidate_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "650b258cc1b74350876a6c305b230d59"
I20260811 14:34:38.833246 13380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd871ae5d41e4599968f6c630137fa5a" candidate_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6"
I20260811 14:34:38.833331 13530 raft_consensus.cc:3037] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:38.833331 13380 raft_consensus.cc:3037] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:38.834297 13380 raft_consensus.cc:2454] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5212a9e0df64b8fb4a9d3b8d06b3bf4 in term 1.
I20260811 14:34:38.834297 13530 raft_consensus.cc:2454] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5212a9e0df64b8fb4a9d3b8d06b3bf4 in term 1.
I20260811 14:34:38.834499 13152 leader_election.cc:302] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4dbe50bc0ee34a348eb0be8a93a52ad6, b5212a9e0df64b8fb4a9d3b8d06b3bf4; no voters:
I20260811 14:34:38.834612 13611 raft_consensus.cc:2781] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:38.834784 13611 raft_consensus.cc:686] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [term 1 LEADER]: Becoming Leader. State: Replica: b5212a9e0df64b8fb4a9d3b8d06b3bf4, State: Running, Role: LEADER
I20260811 14:34:38.834892 13611 consensus_queue.cc:237] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } }
I20260811 14:34:38.835788 13059 catalog_manager.cc:5495] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5212a9e0df64b8fb4a9d3b8d06b3bf4 (127.23.186.193). New cstate: current_term: 1 leader_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5212a9e0df64b8fb4a9d3b8d06b3bf4" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 36155 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 } health_report { overall_health: UNKNOWN } } }
I20260811 14:34:38.848328 13601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.845918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.23.186.193:36155)
0811 14:34:38.845989 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:38.845991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:38.846002 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:34:38.846056 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:38.846062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:38.846081 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:38.846162 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:34:38.846303 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.846788 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.847534 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:38.847738 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:38.847811 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:38.847984 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:34:38.848078 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:34:38.848195 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:38.848228 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 14:34:38.848240 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 14:34:38.850303 13530 raft_consensus.cc:1261] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 1 FOLLOWER]: Refusing update from remote peer b5212a9e0df64b8fb4a9d3b8d06b3bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:38.850289 13380 raft_consensus.cc:1261] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 1 FOLLOWER]: Refusing update from remote peer b5212a9e0df64b8fb4a9d3b8d06b3bf4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:38.850713 13611 consensus_queue.cc:1035] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbe50bc0ee34a348eb0be8a93a52ad6" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 45549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:38.850869 13615 consensus_queue.cc:1035] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "650b258cc1b74350876a6c305b230d59" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 40355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:38.853826 13618 mvcc.cc:205] Tried to move back new op lower bound from 7317335567767965696 to 7317335567708536832. Current Snapshot: MvccSnapshot[applied={T|T < 7317335567767965696 or (T in {7317335567767965696})}]
I20260811 14:34:38.855997 13058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:38.856118 13058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40050:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:38.856911 13058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cbf8eb014d341538e9654fa0a048229: Sending DeleteTablet for 3 replicas of tablet bd871ae5d41e4599968f6c630137fa5a
I20260811 14:34:38.857278 13206 tablet_service.cc:1514] Processing DeleteTablet for tablet bd871ae5d41e4599968f6c630137fa5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:38 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:36786
I20260811 14:34:38.857307 13357 tablet_service.cc:1514] Processing DeleteTablet for tablet bd871ae5d41e4599968f6c630137fa5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:38 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:48478
I20260811 14:34:38.857317 13508 tablet_service.cc:1514] Processing DeleteTablet for tablet bd871ae5d41e4599968f6c630137fa5a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:38 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:47720
I20260811 14:34:38.857623 13629 tablet_replica.cc:330] T bd871ae5d41e4599968f6c630137fa5a P b5212a9e0df64b8fb4a9d3b8d06b3bf4: stopping tablet replica
I20260811 14:34:38.857853 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13131
I20260811 14:34:38.860855 13631 tablet_replica.cc:330] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: stopping tablet replica
I20260811 14:34:38.861029 13631 raft_consensus.cc:2229] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:38.861200 13631 raft_consensus.cc:2258] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:38.861423 13630 tablet_replica.cc:330] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: stopping tablet replica
I20260811 14:34:38.861549 13630 raft_consensus.cc:2229] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:38.861752 13630 raft_consensus.cc:2258] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:38.862375 13630 ts_tablet_manager.cc:1905] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:38.864668 13630 ts_tablet_manager.cc:1918] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:38.864789 13630 log.cc:1196] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/bd871ae5d41e4599968f6c630137fa5a
I20260811 14:34:38.865134 13630 ts_tablet_manager.cc:1939] T bd871ae5d41e4599968f6c630137fa5a P 4dbe50bc0ee34a348eb0be8a93a52ad6: Deleting consensus metadata
I20260811 14:34:38.865275 13631 ts_tablet_manager.cc:1905] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:38.865684 13041 catalog_manager.cc:4841] TS 4dbe50bc0ee34a348eb0be8a93a52ad6 (127.23.186.194:45549): tablet bd871ae5d41e4599968f6c630137fa5a (table test-table [id=ad2e6b2c24af4fcd8241c2b0fb0f52d9]) successfully deleted
I20260811 14:34:38.867086 13631 ts_tablet_manager.cc:1918] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:38.867163 13631 log.cc:1196] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/bd871ae5d41e4599968f6c630137fa5a
I20260811 14:34:38.867452 13631 ts_tablet_manager.cc:1939] T bd871ae5d41e4599968f6c630137fa5a P 650b258cc1b74350876a6c305b230d59: Deleting consensus metadata
I20260811 14:34:38.867904 13039 catalog_manager.cc:4841] TS 650b258cc1b74350876a6c305b230d59 (127.23.186.195:40355): tablet bd871ae5d41e4599968f6c630137fa5a (table test-table [id=ad2e6b2c24af4fcd8241c2b0fb0f52d9]) successfully deleted
W20260811 14:34:38.868049 13040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:36155 (error 108)
I20260811 14:34:38.868371 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13284
I20260811 14:34:38.869824 13596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:38.868445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:36155 (local address 127.0.0.1:36804)
0811 14:34:38.869726 (+ 1281us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:38.869775 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:36155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1254}
W20260811 14:34:38.870940 13040 catalog_manager.cc:4584] TS b5212a9e0df64b8fb4a9d3b8d06b3bf4 (127.23.186.193:36155): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bd871ae5d41e4599968f6c630137fa5a: Network error: Client connection negotiation failed: client connection to 127.23.186.193:36155: connect: Connection refused (error 111)
I20260811 14:34:38.876302 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13437
I20260811 14:34:38.883548 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13022
2026-08-11T14:34:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2166 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T14:34:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:38Z Disabled control of system clock
I20260811 14:34:38.916121 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:36293
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:39.005025 13637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:39.005362 13637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:39.005400 13637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:39.007129 13637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:39.007181 13637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:39.007195 13637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:39.007208 13637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:39.007251 13637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:39.008749 13637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:36293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:39.009032 13637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:39.009346 13637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:39.014360 13648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:39.014333 13646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:39.014508 13637 server_base.cc:1034] running on GCE node
W20260811 14:34:39.014349 13645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:39.014840 13637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:39.015100 13637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:39.016285 13637 hybrid_clock.cc:648] HybridClock initialized: now 1786458879016265 us; error 42 us; skew 500 ppm
I20260811 14:34:39.023815 13637 webserver.cc:460] Webserver started at http://127.23.186.254:41147/ using document root <none> and password file <none>
I20260811 14:34:39.024137 13637 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:39.024192 13637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:39.024381 13637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:39.025622 13637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "6f3e45c0cc8a493bb64934d57ab39c9e"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "773028d2b4d03e46a9da0b2d1939613e"
server_key_iv: "c87efe7b6a571d2486c73c61a720a2b1"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.026012 13637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "6f3e45c0cc8a493bb64934d57ab39c9e"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "773028d2b4d03e46a9da0b2d1939613e"
server_key_iv: "c87efe7b6a571d2486c73c61a720a2b1"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.027390 13637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:34:39.028185 13655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.028393 13637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:39.028505 13637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "6f3e45c0cc8a493bb64934d57ab39c9e"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "773028d2b4d03e46a9da0b2d1939613e"
server_key_iv: "c87efe7b6a571d2486c73c61a720a2b1"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.028582 13637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:39.049197 13637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:39.049587 13637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:39.049764 13637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:39.054057 13637 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:36293
I20260811 14:34:39.054096 13714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:36293 every 8 connection(s)
I20260811 14:34:39.054486 13637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:39.055546 13715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:39.058214 13715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e: Bootstrap starting.
I20260811 14:34:39.058853 13715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:39.059172 13715 log.cc:824] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:39.059891 13715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e: No bootstrap required, opened a new log
I20260811 14:34:39.061610 13715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36293 } }
I20260811 14:34:39.061785 13715 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:39.061836 13715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f3e45c0cc8a493bb64934d57ab39c9e, State: Initialized, Role: FOLLOWER
I20260811 14:34:39.061940 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13637
I20260811 14:34:39.062006 13715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36293 } }
I20260811 14:34:39.062072 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:39.062106 13715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:39.062155 13715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:39.062204 13715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:39.062310 24299 external_mini_cluster.cc:1428] Setting key 5d1a02f89efa146c83f0210733134b14
I20260811 14:34:39.062891 13715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36293 } }
I20260811 14:34:39.063015 13715 leader_election.cc:302] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f3e45c0cc8a493bb64934d57ab39c9e; no voters:
I20260811 14:34:39.063221 13715 leader_election.cc:288] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:39.063271 13720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:39.063410 13720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [term 1 LEADER]: Becoming Leader. State: Replica: 6f3e45c0cc8a493bb64934d57ab39c9e, State: Running, Role: LEADER
I20260811 14:34:39.063510 13715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:39.063520 13720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36293 } }
I20260811 14:34:39.064059 13720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f3e45c0cc8a493bb64934d57ab39c9e. Latest consensus state: current_term: 1 leader_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36293 } } }
I20260811 14:34:39.064172 13720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:39.064479 13721 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f3e45c0cc8a493bb64934d57ab39c9e" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 36293 } } }
I20260811 14:34:39.064586 13721 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:39.064829 13727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:39.065416 13727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:39.067008 13727 catalog_manager.cc:1347] Generated new cluster ID: fa9b1f7326624ba3ae93b836bc584b5b
I20260811 14:34:39.067061 13727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:39.069551 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.062875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36950 (local address 127.23.186.254:36293)
0811 14:34:39.063150 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.063159 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.063195 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:34:39.063344 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.063352 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.063438 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.063871 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:39.063979 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.066071 (+ 2092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.067276 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.067877 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.068003 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.068799 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:39.068803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:39.068803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:39.069029 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:39.069069 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:39.069083 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:39.069148 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.069266 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.069346 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":125,"threads_started":1}
I20260811 14:34:39.089788 13727 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:39.090546 13727 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:39.094362 13727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e: Generated new TSK 0
I20260811 14:34:39.094570 13727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:39.101303 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36293
--builtin_ntp_servers=127.23.186.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:39.211709 13741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:39.211926 13741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:39.211952 13741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:39.213891 13741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:39.213972 13741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:39.214026 13741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:39.215716 13741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:39.216008 13741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:39.216339 13741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:39.221900 13747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:39.221900 13748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:39.221976 13750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:39.222438 13741 server_base.cc:1034] running on GCE node
I20260811 14:34:39.222675 13741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:39.222966 13741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:39.224148 13741 hybrid_clock.cc:648] HybridClock initialized: now 1786458879224143 us; error 40 us; skew 500 ppm
I20260811 14:34:39.225564 13741 webserver.cc:460] Webserver started at http://127.23.186.193:41047/ using document root <none> and password file <none>
I20260811 14:34:39.225852 13741 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:39.225921 13741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:39.226070 13741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:39.227191 13741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "34bd7128728940c0a2ca3ec50e27e2e5"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "f472f12373da7220285f61ec8a0c6949"
server_key_iv: "b7cb3744f9d06e589b5bae51c27247fe"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.227567 13741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "34bd7128728940c0a2ca3ec50e27e2e5"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "f472f12373da7220285f61ec8a0c6949"
server_key_iv: "b7cb3744f9d06e589b5bae51c27247fe"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.229135 13741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:39.229893 13756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.230075 13741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:39.230187 13741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "34bd7128728940c0a2ca3ec50e27e2e5"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "f472f12373da7220285f61ec8a0c6949"
server_key_iv: "b7cb3744f9d06e589b5bae51c27247fe"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.230262 13741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:39.255146 13741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:39.255532 13741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:39.255707 13741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:39.255962 13741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:39.256330 13741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:39.256384 13741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.256428 13741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:39.256466 13741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.262822 13741 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:33201
I20260811 14:34:39.262899 13889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:33201 every 8 connection(s)
I20260811 14:34:39.263283 13741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:39.268743 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13741
I20260811 14:34:39.268868 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:34:39.269063 24299 external_mini_cluster.cc:1428] Setting key de58db0959f0580a02754bc6a0264363
I20260811 14:34:39.269320 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.264161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:52353 (local address 127.23.186.254:36293)
0811 14:34:39.264212 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.264215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.264502 (+ 287us) server_negotiation.cc:407] Connection header received
0811 14:34:39.265185 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.265186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.265203 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.265269 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:39.266084 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.266618 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.267770 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.267970 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.268028 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.269009 (+ 981us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:39.269010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:39.269010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:39.269016 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:39.269032 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:39.269036 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:39.269102 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.269211 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.269252 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:34:39.269585 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.264162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:36293 (local address 127.23.186.193:52353)
0811 14:34:39.264368 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.264404 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.265057 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.265347 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.265351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.265368 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:39.265982 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.265987 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.266782 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.266783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.267677 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.267681 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.268051 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.268052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.268173 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.268824 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:39.268824 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:39.268874 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:39.268906 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:39.269103 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:39.269121 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:39.269124 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.269204 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.269331 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260811 14:34:39.270074 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36293
--builtin_ntp_servers=127.23.186.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:34:39.270272 13890 heartbeater.cc:347] Connected to a master server at 127.23.186.254:36293
I20260811 14:34:39.270390 13890 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:39.270578 13890 heartbeater.cc:511] Master 127.23.186.254:36293 requested a full tablet report, sending...
I20260811 14:34:39.271099 13673 ts_manager.cc:194] Registered new tserver with Master: 34bd7128728940c0a2ca3ec50e27e2e5 (127.23.186.193:33201)
I20260811 14:34:39.271739 13673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.193:52353
W20260811 14:34:39.354387 13893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:39.354604 13893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:39.354626 13893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:39.355794 13893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:39.355837 13893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:39.355865 13893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:39.357370 13893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:39.357640 13893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:39.357963 13893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:39.363204 13898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:39.363214 13902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:39.363497 13893 server_base.cc:1034] running on GCE node
W20260811 14:34:39.363560 13900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:39.363821 13893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:39.364094 13893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:39.365274 13893 hybrid_clock.cc:648] HybridClock initialized: now 1786458879365251 us; error 33 us; skew 500 ppm
I20260811 14:34:39.366683 13893 webserver.cc:460] Webserver started at http://127.23.186.194:44309/ using document root <none> and password file <none>
I20260811 14:34:39.367041 13893 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:39.367115 13893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:39.367264 13893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:39.368395 13893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "d0f616b21efd47209620d3ddd84460a9"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "ba4ebcfae0923b468c22e99aea6b0936"
server_key_iv: "9b62a5822de0f06732564b0422f0b5e4"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.368839 13893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "d0f616b21efd47209620d3ddd84460a9"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "ba4ebcfae0923b468c22e99aea6b0936"
server_key_iv: "9b62a5822de0f06732564b0422f0b5e4"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.370262 13893 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:39.371081 13909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.371268 13893 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:34:39.371372 13893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "d0f616b21efd47209620d3ddd84460a9"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "ba4ebcfae0923b468c22e99aea6b0936"
server_key_iv: "9b62a5822de0f06732564b0422f0b5e4"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.371461 13893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:39.385545 13893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:39.385885 13893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:39.386044 13893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:39.386299 13893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:39.386641 13893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:39.386696 13893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.386737 13893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:39.386762 13893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.392825 13893 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:43533
I20260811 14:34:39.392879 14041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:43533 every 8 connection(s)
I20260811 14:34:39.393239 13893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:34:39.396345 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 13893
I20260811 14:34:39.396456 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:34:39.396649 24299 external_mini_cluster.cc:1428] Setting key 906496d0cab8116ca608c3b0c041231c
I20260811 14:34:39.397763 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36293
--builtin_ntp_servers=127.23.186.212:44541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 14:34:39.399256 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.394005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:44007 (local address 127.23.186.254:36293)
0811 14:34:39.394093 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.394096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.394290 (+ 194us) server_negotiation.cc:407] Connection header received
0811 14:34:39.394809 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.394810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.394821 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.394875 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:39.395587 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.396089 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.397572 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.397898 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.397973 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.398979 (+ 1006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:39.398980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:39.398980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:39.398986 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:39.399003 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:39.399008 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:39.399054 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.399168 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.399196 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 14:34:39.399569 14044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.393953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:36293 (local address 127.23.186.194:44007)
0811 14:34:39.394166 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.394203 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.394716 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.394955 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.394959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.394967 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:39.395510 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.395514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.396303 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.396304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.397478 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.397481 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.398051 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.398052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.398232 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.398813 (+ 581us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:39.398813 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:39.398854 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:39.398886 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:39.399069 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:39.399083 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:39.399086 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.399165 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.399287 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
I20260811 14:34:39.400058 14042 heartbeater.cc:347] Connected to a master server at 127.23.186.254:36293
I20260811 14:34:39.400172 14042 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:39.400377 14042 heartbeater.cc:511] Master 127.23.186.254:36293 requested a full tablet report, sending...
I20260811 14:34:39.400827 13673 ts_manager.cc:194] Registered new tserver with Master: d0f616b21efd47209620d3ddd84460a9 (127.23.186.194:43533)
I20260811 14:34:39.401221 13673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.194:44007
W20260811 14:34:39.484489 14047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:39.484826 14047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:39.484876 14047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:39.486133 14047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:39.486182 14047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:39.486212 14047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:39.487655 14047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:39.487941 14047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:39.488276 14047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:39.493686 14055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:39.493643 14059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:39.493673 14057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:39.493938 14047 server_base.cc:1034] running on GCE node
I20260811 14:34:39.494158 14047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:39.494391 14047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:39.495572 14047 hybrid_clock.cc:648] HybridClock initialized: now 1786458879495551 us; error 38 us; skew 500 ppm
I20260811 14:34:39.497031 14047 webserver.cc:460] Webserver started at http://127.23.186.195:44809/ using document root <none> and password file <none>
I20260811 14:34:39.497349 14047 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:39.497402 14047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:39.497532 14047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:39.498700 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "a86c7663e54c49fd8eab1fea8cfaa698"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "c207e7ca88cb04c8a97bcad6b1bc91f3"
server_key_iv: "d6607ea1f949da9825a86c34b022562a"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.499106 14047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "a86c7663e54c49fd8eab1fea8cfaa698"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "c207e7ca88cb04c8a97bcad6b1bc91f3"
server_key_iv: "d6607ea1f949da9825a86c34b022562a"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.500608 14047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:39.501534 14066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.501713 14047 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:39.501814 14047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "a86c7663e54c49fd8eab1fea8cfaa698"
format_stamp: "Formatted at 2026-08-11 14:34:39 on dist-test-slave-cbsf"
server_key: "c207e7ca88cb04c8a97bcad6b1bc91f3"
server_key_iv: "d6607ea1f949da9825a86c34b022562a"
server_key_version: "encryptionkey@0"
I20260811 14:34:39.501896 14047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:39.518994 14047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:39.519359 14047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:39.519533 14047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:39.519795 14047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:39.520160 14047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:39.520213 14047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.520251 14047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:39.520292 14047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:39.526327 14047 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:43875
I20260811 14:34:39.526402 14200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:43875 every 8 connection(s)
I20260811 14:34:39.526783 14047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:39.532630 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.527668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:60131 (local address 127.23.186.254:36293)
0811 14:34:39.527760 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.527763 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.527983 (+ 220us) server_negotiation.cc:407] Connection header received
0811 14:34:39.528598 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.528600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.528631 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.528714 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:39.529456 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.529991 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.531031 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.531239 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.531318 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.532295 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:39.532297 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:39.532297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:39.532303 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:39.532321 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:39.532326 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:39.532392 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.532474 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.532513 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 14:34:39.532853 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.527613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:36293 (local address 127.23.186.195:60131)
0811 14:34:39.527861 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.527898 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.528484 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.528797 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.528802 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.528812 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:39.529357 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.529360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.530093 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.530094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.530918 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.530920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.531332 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.531333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.531481 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.532126 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:39.532126 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:39.532158 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:39.532185 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:39.532393 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:39.532400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:39.532404 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.532470 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.532585 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":105,"threads_started":1}
I20260811 14:34:39.533429 14201 heartbeater.cc:347] Connected to a master server at 127.23.186.254:36293
I20260811 14:34:39.533552 14201 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:39.533751 14201 heartbeater.cc:511] Master 127.23.186.254:36293 requested a full tablet report, sending...
I20260811 14:34:39.534173 13673 ts_manager.cc:194] Registered new tserver with Master: a86c7663e54c49fd8eab1fea8cfaa698 (127.23.186.195:43875)
I20260811 14:34:39.534557 13673 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.195:60131
I20260811 14:34:39.535182 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14047
I20260811 14:34:39.535274 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:34:39.535490 24299 external_mini_cluster.cc:1428] Setting key e82dcde0a2e12ee28351e0fc9b96bbd9
I20260811 14:34:39.536398 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:34:39.540524 13719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.537817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36960 (local address 127.23.186.254:36293)
0811 14:34:39.537891 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.537893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.537948 (+ 55us) server_negotiation.cc:407] Connection header received
0811 14:34:39.537994 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.537994 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.538007 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.538054 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:39.538189 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.538654 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.539343 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.539519 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.539566 (+ 47us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.540218 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:39.540219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:39.540219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:39.540224 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:39.540238 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:39.540242 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:39.540318 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.540391 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.540419 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:34:39.542164 13673 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:34:39.543813 13673 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:34:39.550297 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.546019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.23.186.195:43875)
0811 14:34:39.546579 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.546583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.546593 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:39.546632 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.546633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.546661 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.546921 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:39.546986 (+ 65us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.547696 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.549124 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.549858 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.549940 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.550036 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.550151 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.550201 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":96,"threads_started":1}
I20260811 14:34:39.550305 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.545924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:43875 (local address 127.0.0.1:36864)
0811 14:34:39.546087 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.546134 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.546222 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.546778 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.546779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.546782 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:39.546842 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.546844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.547793 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.547794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.549026 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.549029 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.549952 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.549954 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.550041 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.550061 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.550138 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.550206 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":29,"thread_start_us":114,"threads_started":1}
I20260811 14:34:39.550441 14210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.545937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43533 (local address 127.0.0.1:56778)
0811 14:34:39.546320 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.546330 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.546374 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.546728 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.546731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.546734 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:39.546841 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.546843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.548264 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.548266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.549291 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.549294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.550212 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.550213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.550277 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.550281 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.550313 (+ 32us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.550358 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":4,"thread_start_us":94,"threads_started":1}
I20260811 14:34:39.550550 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.546341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.23.186.193:33201)
0811 14:34:39.546520 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.546525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.546535 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:39.546582 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.546583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.546608 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.546891 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:39.546962 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.548132 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.549361 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.549869 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.550264 (+ 395us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.550365 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.550394 (+ 29us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.550443 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260811 14:34:39.550835 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.546360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.23.186.194:43533)
0811 14:34:39.546533 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.546537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.546548 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:34:39.546584 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.546585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.546599 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.546995 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:39.547059 (+ 64us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.548160 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.549395 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.550111 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.550549 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.550663 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.550691 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.550730 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":90,"threads_started":1}
I20260811 14:34:39.551174 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.546193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:33201 (local address 127.0.0.1:33280)
0811 14:34:39.546344 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.546353 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.546395 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.546728 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.546731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.546736 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:39.546910 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.546911 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.548229 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.548231 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.549277 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.549280 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.549987 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.549987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.550041 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.550061 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.550122 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.550206 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":107,"thread_start_us":90,"threads_started":1}
I20260811 14:34:39.551597 14121 tablet_service.cc:1467] Processing CreateTablet for tablet c4bb6efb441847e39bfd3999a273c793 (DEFAULT_TABLE table=test-table [id=89036a4f03d44d44b3ac60c0f1f682fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:39.551932 13806 tablet_service.cc:1467] Processing CreateTablet for tablet c4bb6efb441847e39bfd3999a273c793 (DEFAULT_TABLE table=test-table [id=89036a4f03d44d44b3ac60c0f1f682fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:39.552048 14121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb6efb441847e39bfd3999a273c793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:39.552273 13806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb6efb441847e39bfd3999a273c793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:39.553045 13964 tablet_service.cc:1467] Processing CreateTablet for tablet c4bb6efb441847e39bfd3999a273c793 (DEFAULT_TABLE table=test-table [id=89036a4f03d44d44b3ac60c0f1f682fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:39.553355 13964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4bb6efb441847e39bfd3999a273c793. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:39.555675 14217 tablet_bootstrap.cc:492] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: Bootstrap starting.
I20260811 14:34:39.556131 14219 tablet_bootstrap.cc:492] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Bootstrap starting.
I20260811 14:34:39.556357 14217 tablet_bootstrap.cc:654] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:39.556595 14217 log.cc:824] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:39.556824 14219 tablet_bootstrap.cc:654] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:39.557006 14218 tablet_bootstrap.cc:492] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Bootstrap starting.
I20260811 14:34:39.557165 14219 log.cc:824] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:39.557464 14217 tablet_bootstrap.cc:492] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: No bootstrap required, opened a new log
I20260811 14:34:39.557545 14217 ts_tablet_manager.cc:1397] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:34:39.557788 14218 tablet_bootstrap.cc:654] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:39.558058 14219 tablet_bootstrap.cc:492] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: No bootstrap required, opened a new log
I20260811 14:34:39.558151 14219 ts_tablet_manager.cc:1397] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:34:39.558175 14218 log.cc:824] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:39.559020 14218 tablet_bootstrap.cc:492] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: No bootstrap required, opened a new log
I20260811 14:34:39.559100 14218 ts_tablet_manager.cc:1397] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:34:39.560370 14219 raft_consensus.cc:348] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.560359 14217 raft_consensus.cc:348] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.560518 14217 raft_consensus.cc:374] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:39.560527 14219 raft_consensus.cc:374] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:39.560555 14217 raft_consensus.cc:729] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34bd7128728940c0a2ca3ec50e27e2e5, State: Initialized, Role: FOLLOWER
I20260811 14:34:39.560572 14219 raft_consensus.cc:729] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0f616b21efd47209620d3ddd84460a9, State: Initialized, Role: FOLLOWER
I20260811 14:34:39.560727 14219 consensus_queue.cc:260] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.560735 14217 consensus_queue.cc:260] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.561009 14219 ts_tablet_manager.cc:1428] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 14:34:39.561015 14217 ts_tablet_manager.cc:1428] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:34:39.561194 13890 heartbeater.cc:503] Master 127.23.186.254:36293 was elected leader, sending a full tablet report...
I20260811 14:34:39.561491 14042 heartbeater.cc:503] Master 127.23.186.254:36293 was elected leader, sending a full tablet report...
I20260811 14:34:39.561825 14218 raft_consensus.cc:348] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.562000 14218 raft_consensus.cc:374] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:39.562041 14218 raft_consensus.cc:729] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a86c7663e54c49fd8eab1fea8cfaa698, State: Initialized, Role: FOLLOWER
I20260811 14:34:39.562180 14218 consensus_queue.cc:260] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.562455 14218 ts_tablet_manager.cc:1428] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:34:39.562551 14201 heartbeater.cc:503] Master 127.23.186.254:36293 was elected leader, sending a full tablet report...
W20260811 14:34:39.644007 14043 tablet.cc:2369] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:34:39.680948 14226 raft_consensus.cc:482] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:39.681201 14226 raft_consensus.cc:504] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.681609 14226 leader_election.cc:288] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0f616b21efd47209620d3ddd84460a9 (127.23.186.194:43533), 34bd7128728940c0a2ca3ec50e27e2e5 (127.23.186.193:33201)
I20260811 14:34:39.685817 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.682042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:51207 (local address 127.23.186.193:33201)
0811 14:34:39.682115 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.682117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.682229 (+ 112us) server_negotiation.cc:407] Connection header received
0811 14:34:39.682302 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.682303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.682320 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.682360 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:39.682527 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.683216 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.684850 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.685426 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.685512 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.685521 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.685695 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.685739 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:34:39.685817 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.682026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:56111 (local address 127.23.186.194:43533)
0811 14:34:39.682076 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.682078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.682090 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:34:39.682132 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.682133 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.682144 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.682243 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:39.682430 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.683216 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.684850 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.685426 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.685512 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.685521 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.685695 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.685739 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:34:39.685839 14203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.681936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43533 (local address 127.23.186.195:56111)
0811 14:34:39.682007 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.682021 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.682068 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.682248 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.682249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.682251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:39.682331 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.682332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.683314 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.683315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.684735 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.684740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.685519 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.685520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.685606 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.685611 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.685673 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.685738 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":60}
I20260811 14:34:39.685853 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.681938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:33201 (local address 127.23.186.195:51207)
0811 14:34:39.682146 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.682155 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:39.682233 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:39.682384 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:39.682385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:39.682387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:39.682448 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.682450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.683314 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.683315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.684735 (+ 1420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:39.684740 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.685519 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.685520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:39.685606 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.685617 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 14:34:39.685673 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 14:34:39.685737 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":89,"thread_start_us":114,"threads_started":1}
I20260811 14:34:39.686343 13986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb6efb441847e39bfd3999a273c793" candidate_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f616b21efd47209620d3ddd84460a9" is_pre_election: true
I20260811 14:34:39.686398 13831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb6efb441847e39bfd3999a273c793" candidate_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" is_pre_election: true
I20260811 14:34:39.686550 13986 raft_consensus.cc:2454] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a86c7663e54c49fd8eab1fea8cfaa698 in term 0.
I20260811 14:34:39.686549 13831 raft_consensus.cc:2454] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a86c7663e54c49fd8eab1fea8cfaa698 in term 0.
I20260811 14:34:39.686816 14069 leader_election.cc:302] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34bd7128728940c0a2ca3ec50e27e2e5, a86c7663e54c49fd8eab1fea8cfaa698; no voters:
I20260811 14:34:39.686972 14226 raft_consensus.cc:2781] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:34:39.687034 14226 raft_consensus.cc:482] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:34:39.687059 14226 raft_consensus.cc:3037] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:39.687826 14226 raft_consensus.cc:504] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.687994 14226 leader_election.cc:288] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [CANDIDATE]: Term 1 election: Requested vote from peers d0f616b21efd47209620d3ddd84460a9 (127.23.186.194:43533), 34bd7128728940c0a2ca3ec50e27e2e5 (127.23.186.193:33201)
I20260811 14:34:39.688203 13831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb6efb441847e39bfd3999a273c793" candidate_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bd7128728940c0a2ca3ec50e27e2e5"
I20260811 14:34:39.688221 13986 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4bb6efb441847e39bfd3999a273c793" candidate_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0f616b21efd47209620d3ddd84460a9"
I20260811 14:34:39.688309 13986 raft_consensus.cc:3037] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:39.688310 13831 raft_consensus.cc:3037] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:39.689373 13986 raft_consensus.cc:2454] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a86c7663e54c49fd8eab1fea8cfaa698 in term 1.
I20260811 14:34:39.689379 13831 raft_consensus.cc:2454] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a86c7663e54c49fd8eab1fea8cfaa698 in term 1.
I20260811 14:34:39.689575 14069 leader_election.cc:302] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34bd7128728940c0a2ca3ec50e27e2e5, a86c7663e54c49fd8eab1fea8cfaa698; no voters:
I20260811 14:34:39.689695 14226 raft_consensus.cc:2781] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:39.689890 14226 raft_consensus.cc:686] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 1 LEADER]: Becoming Leader. State: Replica: a86c7663e54c49fd8eab1fea8cfaa698, State: Running, Role: LEADER
I20260811 14:34:39.690009 14226 consensus_queue.cc:237] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } }
I20260811 14:34:39.690984 13671 catalog_manager.cc:5495] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 reported cstate change: term changed from 0 to 1, leader changed from <none> to a86c7663e54c49fd8eab1fea8cfaa698 (127.23.186.195). New cstate: current_term: 1 leader_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a86c7663e54c49fd8eab1fea8cfaa698" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 43875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 } health_report { overall_health: UNKNOWN } } }
I20260811 14:34:39.733156 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.730631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36866 (local address 127.23.186.195:43875)
0811 14:34:39.730695 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:39.730697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:39.730703 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:34:39.730740 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:39.730741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:39.730763 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:39.730849 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:34:39.730979 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.731470 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.732283 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:39.732476 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:39.732552 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:39.732739 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:34:39.732866 (+ 127us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:34:39.732970 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:39.733050 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:34:39.733059 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 14:34:39.735163 13986 raft_consensus.cc:1261] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 1 FOLLOWER]: Refusing update from remote peer a86c7663e54c49fd8eab1fea8cfaa698: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:39.735172 13831 raft_consensus.cc:1261] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 1 FOLLOWER]: Refusing update from remote peer a86c7663e54c49fd8eab1fea8cfaa698: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:39.735590 14229 consensus_queue.cc:1035] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0f616b21efd47209620d3ddd84460a9" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:39.735735 14226 consensus_queue.cc:1035] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [LEADER]: Connected to new peer: Peer: permanent_uuid: "34bd7128728940c0a2ca3ec50e27e2e5" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 33201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:39.739343 14233 mvcc.cc:205] Tried to move back new op lower bound from 7317335571392380928 to 7317335571211104256. Current Snapshot: MvccSnapshot[applied={T|T < 7317335571392380928 or (T in {7317335571392380928})}]
I20260811 14:34:39.740788 13672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:39.740897 13672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36960:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:39.741665 13672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f3e45c0cc8a493bb64934d57ab39c9e: Sending DeleteTablet for 3 replicas of tablet c4bb6efb441847e39bfd3999a273c793
I20260811 14:34:39.742002 13964 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb6efb441847e39bfd3999a273c793 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:39 UTC) from {username='slave'} at 127.0.0.1:56778
I20260811 14:34:39.742004 14121 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb6efb441847e39bfd3999a273c793 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:39 UTC) from {username='slave'} at 127.0.0.1:36864
I20260811 14:34:39.742049 13806 tablet_service.cc:1514] Processing DeleteTablet for tablet c4bb6efb441847e39bfd3999a273c793 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:39 UTC) from {username='slave'} at 127.0.0.1:33280
I20260811 14:34:39.742249 14248 tablet_replica.cc:330] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: stopping tablet replica
I20260811 14:34:39.742291 14250 tablet_replica.cc:330] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5: stopping tablet replica
I20260811 14:34:39.742344 14248 raft_consensus.cc:2229] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:34:39.742386 14250 raft_consensus.cc:2229] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:39.742460 14248 raft_consensus.cc:2258] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:39.742482 14250 raft_consensus.cc:2258] T c4bb6efb441847e39bfd3999a273c793 P 34bd7128728940c0a2ca3ec50e27e2e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:39.742594 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13741
I20260811 14:34:39.744270 14248 ts_tablet_manager.cc:1905] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:39.746338 14248 ts_tablet_manager.cc:1918] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:39.746410 14248 log.cc:1196] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/c4bb6efb441847e39bfd3999a273c793
I20260811 14:34:39.746677 14248 ts_tablet_manager.cc:1939] T c4bb6efb441847e39bfd3999a273c793 P a86c7663e54c49fd8eab1fea8cfaa698: Deleting consensus metadata
I20260811 14:34:39.747385 13657 catalog_manager.cc:4841] TS a86c7663e54c49fd8eab1fea8cfaa698 (127.23.186.195:43875): tablet c4bb6efb441847e39bfd3999a273c793 (table test-table [id=89036a4f03d44d44b3ac60c0f1f682fb]) successfully deleted
I20260811 14:34:39.747838 14249 tablet_replica.cc:330] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: stopping tablet replica
I20260811 14:34:39.747983 14249 raft_consensus.cc:2229] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:39.748126 14249 raft_consensus.cc:2258] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:39.748780 14249 ts_tablet_manager.cc:1905] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:39.750566 14249 ts_tablet_manager.cc:1918] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:39.750651 14249 log.cc:1196] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/c4bb6efb441847e39bfd3999a273c793
I20260811 14:34:39.750941 14249 ts_tablet_manager.cc:1939] T c4bb6efb441847e39bfd3999a273c793 P d0f616b21efd47209620d3ddd84460a9: Deleting consensus metadata
I20260811 14:34:39.751624 13659 catalog_manager.cc:4841] TS d0f616b21efd47209620d3ddd84460a9 (127.23.186.194:43533): tablet c4bb6efb441847e39bfd3999a273c793 (table test-table [id=89036a4f03d44d44b3ac60c0f1f682fb]) successfully deleted
W20260811 14:34:39.752771 13657 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:33201 (error 108)
I20260811 14:34:39.753049 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13893
I20260811 14:34:39.754524 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:39.753210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:33201 (local address 127.0.0.1:33294)
0811 14:34:39.754431 (+ 1221us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:39.754479 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:33201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1197}
W20260811 14:34:39.760618 13657 catalog_manager.cc:4584] TS 34bd7128728940c0a2ca3ec50e27e2e5 (127.23.186.193:33201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4bb6efb441847e39bfd3999a273c793: Network error: Client connection negotiation failed: client connection to 127.23.186.193:33201: connect: Connection refused (error 111)
I20260811 14:34:39.769603 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14047
I20260811 14:34:39.777696 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 13637
2026-08-11T14:34:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (892 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:39 dist-test-slave-cbsf krb5kdc[14255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:39 dist-test-slave-cbsf krb5kdc[14255](info): set up 2 sockets
Aug 11 14:34:39 dist-test-slave-cbsf krb5kdc[14255](info): commencing operation
krb5kdc: starting...
W20260811 14:34:40.888862 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:40 dist-test-slave-cbsf krb5kdc[14255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:34:41.023578 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40351
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:40351
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:41.109040 14270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:41.109324 14270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:41.109365 14270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:41.110569 14270 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:41.110656 14270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:41.110679 14270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:41.110702 14270 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:41.110729 14270 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:41.112305 14270 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:40351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:41.112634 14270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:41.112915 14270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:41.118114 14278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.118299 14270 server_base.cc:1034] running on GCE node
W20260811 14:34:41.118407 14276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.118464 14275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.118754 14270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:41.119079 14270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:41.120246 14270 hybrid_clock.cc:648] HybridClock initialized: now 1786458881120231 us; error 40 us; skew 500 ppm
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:41.123054 14270 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:34:41.123580 14270 webserver.cc:460] Webserver started at http://127.23.186.254:38511/ using document root <none> and password file <none>
I20260811 14:34:41.123832 14270 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:41.123899 14270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:41.124053 14270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:41.125276 14270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "c7d23578e99245c3aea4a52d00ed7481"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "81ff7bba2e75bc50ed6dbd99b6502548"
server_key_iv: "a7b49372d9b69c8b91eccf71e5ee9e9e"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.125672 14270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "c7d23578e99245c3aea4a52d00ed7481"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "81ff7bba2e75bc50ed6dbd99b6502548"
server_key_iv: "a7b49372d9b69c8b91eccf71e5ee9e9e"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.127141 14270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:34:41.127964 14285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.128152 14270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:41.128265 14270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "c7d23578e99245c3aea4a52d00ed7481"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "81ff7bba2e75bc50ed6dbd99b6502548"
server_key_iv: "a7b49372d9b69c8b91eccf71e5ee9e9e"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.128343 14270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:41.145910 14270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:41.148491 14270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:41.148684 14270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:41.153070 14270 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:40351
I20260811 14:34:41.153117 14344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:40351 every 8 connection(s)
I20260811 14:34:41.153523 14270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:41.154495 14345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:41.156908 14345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481: Bootstrap starting.
I20260811 14:34:41.157497 14345 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:41.157743 14345 log.cc:824] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:41.158461 14345 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481: No bootstrap required, opened a new log
I20260811 14:34:41.159814 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14270
I20260811 14:34:41.159963 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:41.160166 24299 external_mini_cluster.cc:1428] Setting key abd55190045f967ac74797b39c7a0f62
I20260811 14:34:41.160159 14345 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d23578e99245c3aea4a52d00ed7481" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40351 } }
I20260811 14:34:41.160374 14345 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:41.160426 14345 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7d23578e99245c3aea4a52d00ed7481, State: Initialized, Role: FOLLOWER
I20260811 14:34:41.160593 14345 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d23578e99245c3aea4a52d00ed7481" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40351 } }
I20260811 14:34:41.160676 14345 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:41.160745 14345 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:41.160792 14345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:41.161506 14345 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d23578e99245c3aea4a52d00ed7481" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40351 } }
I20260811 14:34:41.161635 14345 leader_election.cc:302] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7d23578e99245c3aea4a52d00ed7481; no voters:
I20260811 14:34:41.161825 14345 leader_election.cc:288] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:41.161988 14353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:41.162117 14345 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:41.162158 14353 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [term 1 LEADER]: Becoming Leader. State: Replica: c7d23578e99245c3aea4a52d00ed7481, State: Running, Role: LEADER
I20260811 14:34:41.162348 14353 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d23578e99245c3aea4a52d00ed7481" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40351 } }
I20260811 14:34:41.162751 14353 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7d23578e99245c3aea4a52d00ed7481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d23578e99245c3aea4a52d00ed7481" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40351 } } }
I20260811 14:34:41.162930 14353 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:41.162809 14352 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7d23578e99245c3aea4a52d00ed7481. Latest consensus state: current_term: 1 leader_uuid: "c7d23578e99245c3aea4a52d00ed7481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7d23578e99245c3aea4a52d00ed7481" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40351 } } }
I20260811 14:34:41.163187 14352 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:41.163203 14361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:41.163761 14361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:41.165539 14361 catalog_manager.cc:1347] Generated new cluster ID: 24870534135a40b28884bc7f81e9a1ee
I20260811 14:34:41.165592 14361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:41.173475 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.160755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58456 (local address 127.23.186.254:40351)
0811 14:34:41.160989 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.161000 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.161038 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:34:41.161473 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.161482 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.161544 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.161688 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:41.162710 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.164255 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.165366 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.165974 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.166077 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.169107 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:41.169112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:41.169113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:41.169128 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:41.171381 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.171994 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.171998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.171999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.172146 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.172361 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.172362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.172362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.172793 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:41.172974 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.173064 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.173229 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":117,"threads_started":1}
I20260811 14:34:41.183537 14361 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:41.184170 14361 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:41.190552 14361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481: Generated new TSK 0
I20260811 14:34:41.190801 14361 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.23.186.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
Entry for principal kudu/127.23.186.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab.
I20260811 14:34:41.227550 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:40351
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:41.316210 14377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:41.316428 14377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:41.316455 14377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:41.317719 14377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:41.317770 14377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:41.317803 14377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:41.319352 14377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.193.keytab
--principal=kudu/127.23.186.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:40351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:41.319631 14377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:41.319872 14377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:41.325182 14383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.325197 14386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.325176 14382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.325630 14377 server_base.cc:1034] running on GCE node
I20260811 14:34:41.325848 14377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:41.326046 14377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:41.327188 14377 hybrid_clock.cc:648] HybridClock initialized: now 1786458881327173 us; error 45 us; skew 500 ppm
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:41.329898 14377 init.cc:377] Logged in from keytab as kudu/127.23.186.193@KRBTEST.COM (short username kudu)
I20260811 14:34:41.330395 14377 webserver.cc:460] Webserver started at http://127.23.186.193:44259/ using document root <none> and password file <none>
I20260811 14:34:41.330683 14377 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:41.330750 14377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:41.330890 14377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:41.332399 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "cc329def3c264d499f5332a4f48ba428"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "48b5c2e4901683c6d23fb7e8add7440b"
server_key_iv: "3015393f588438ef11c6d5717bc7c59f"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.332806 14377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "cc329def3c264d499f5332a4f48ba428"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "48b5c2e4901683c6d23fb7e8add7440b"
server_key_iv: "3015393f588438ef11c6d5717bc7c59f"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.334172 14377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:41.334899 14394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.335071 14377 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:41.335172 14377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "cc329def3c264d499f5332a4f48ba428"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "48b5c2e4901683c6d23fb7e8add7440b"
server_key_iv: "3015393f588438ef11c6d5717bc7c59f"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.335256 14377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:41.349088 14377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:41.351475 14377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:41.351644 14377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:41.351902 14377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:41.352246 14377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:41.352299 14377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.352342 14377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:41.352372 14377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.358572 14377 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:43099
I20260811 14:34:41.358626 14524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:43099 every 8 connection(s)
I20260811 14:34:41.358958 14377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:41.364156 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14377
I20260811 14:34:41.364303 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:34:41.364558 24299 external_mini_cluster.cc:1428] Setting key 629fe8ceba3ca9ecf8159dc287fd6e21
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.193@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:41.370258 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.359681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:58031 (local address 127.23.186.254:40351)
0811 14:34:41.359747 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.359749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.359968 (+ 219us) server_negotiation.cc:407] Connection header received
0811 14:34:41.360476 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.360477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.360489 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.360561 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:41.361755 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.362287 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.363391 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.363689 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.363770 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.366709 (+ 2939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:41.366711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:41.366711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:41.366720 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:41.368555 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.369115 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.369117 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.369117 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.369187 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.369461 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.369462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.369462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.369638 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:41.369709 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.369992 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.370098 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:34:41.370287 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.359650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40351 (local address 127.23.186.193:58031)
0811 14:34:41.359831 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.359874 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.360377 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.360659 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.360663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.361355 (+ 692us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:41.361580 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.361585 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.362426 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.362427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.363283 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.363286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.363780 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.363781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.363894 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.364475 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:41.364485 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:41.366591 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:41.368673 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:41.368681 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:41.368684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:41.369022 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:41.369266 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:41.369267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:41.369267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:41.369385 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:41.369703 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:41.369709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:41.369846 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.369939 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.370081 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":108,"threads_started":1}
I20260811 14:34:41.370995 14525 heartbeater.cc:347] Connected to a master server at 127.23.186.254:40351
I20260811 14:34:41.371119 14525 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:41.371321 14525 heartbeater.cc:511] Master 127.23.186.254:40351 requested a full tablet report, sending...
I20260811 14:34:41.371906 14305 ts_manager.cc:194] Registered new tserver with Master: cc329def3c264d499f5332a4f48ba428 (127.23.186.193:43099)
I20260811 14:34:41.372607 14305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.193@KRBTEST.COM'} at 127.23.186.193:58031
WARNING: no policy specified for kudu/127.23.186.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
Entry for principal kudu/127.23.186.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab.
I20260811 14:34:41.396242 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:40351
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:41.482887 14532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:41.483157 14532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:41.483199 14532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:41.484376 14532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:41.484453 14532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:41.484500 14532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:41.486065 14532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.194.keytab
--principal=kudu/127.23.186.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:40351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14532
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:41.486356 14532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:41.486600 14532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:41.491895 14538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.491948 14537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.492110 14532 server_base.cc:1034] running on GCE node
W20260811 14:34:41.491895 14541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.492383 14532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:41.492654 14532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:41.493844 14532 hybrid_clock.cc:648] HybridClock initialized: now 1786458881493835 us; error 37 us; skew 500 ppm
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:41.496944 14532 init.cc:377] Logged in from keytab as kudu/127.23.186.194@KRBTEST.COM (short username kudu)
I20260811 14:34:41.497452 14532 webserver.cc:460] Webserver started at http://127.23.186.194:40813/ using document root <none> and password file <none>
I20260811 14:34:41.497664 14532 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:41.497709 14532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:41.497805 14532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:41.498904 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "d4d7c7e1f7d043db93a1f19a06645f5f"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "05eb9262b2a4cc46327e9008612ad7b8"
server_key_iv: "b04702d9504984a69c6d4ed7c039fb96"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.499223 14532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "d4d7c7e1f7d043db93a1f19a06645f5f"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "05eb9262b2a4cc46327e9008612ad7b8"
server_key_iv: "b04702d9504984a69c6d4ed7c039fb96"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.500654 14532 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:34:41.501498 14549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.501688 14532 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:41.501797 14532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "d4d7c7e1f7d043db93a1f19a06645f5f"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "05eb9262b2a4cc46327e9008612ad7b8"
server_key_iv: "b04702d9504984a69c6d4ed7c039fb96"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.501878 14532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:41.516139 14532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:41.518721 14532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:41.518901 14532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:41.519165 14532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:41.519503 14532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:41.519558 14532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.519600 14532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:41.519654 14532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.526014 14532 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:43141
I20260811 14:34:41.526062 14678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:43141 every 8 connection(s)
I20260811 14:34:41.526407 14532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.194@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:41.532965 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14532
I20260811 14:34:41.533090 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:34:41.533295 24299 external_mini_cluster.cc:1428] Setting key 2fc1b848988ee66c1854ba224b00fd92
I20260811 14:34:41.536511 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.527135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:50621 (local address 127.23.186.254:40351)
0811 14:34:41.527225 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.527228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.527411 (+ 183us) server_negotiation.cc:407] Connection header received
0811 14:34:41.527900 (+ 489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.527901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.527912 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.527963 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:41.529007 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.529514 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.530542 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.530804 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.530863 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.533350 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:41.533352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:41.533352 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:41.533361 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:41.534900 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.535432 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.535433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.535434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.535491 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.535782 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.535782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.535783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.535896 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:41.535949 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.536275 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.536379 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 14:34:41.536538 14681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.527098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40351 (local address 127.23.186.194:50621)
0811 14:34:41.527283 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.527322 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.527812 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.528025 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.528028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.528639 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:41.528919 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.528924 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.529601 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.529602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.530457 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.530459 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.530882 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.530883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.531005 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.531519 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:41.531529 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:41.533247 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:41.534989 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:41.534998 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:41.535001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:41.535345 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:41.535551 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:41.535551 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:41.535552 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:41.535716 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:41.535961 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:41.535969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:41.536130 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.536223 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.536358 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":104,"threads_started":1}
I20260811 14:34:41.537129 14679 heartbeater.cc:347] Connected to a master server at 127.23.186.254:40351
I20260811 14:34:41.537252 14679 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:41.537458 14679 heartbeater.cc:511] Master 127.23.186.254:40351 requested a full tablet report, sending...
I20260811 14:34:41.537916 14305 ts_manager.cc:194] Registered new tserver with Master: d4d7c7e1f7d043db93a1f19a06645f5f (127.23.186.194:43141)
I20260811 14:34:41.538452 14305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.194@KRBTEST.COM'} at 127.23.186.194:50621
WARNING: no policy specified for kudu/127.23.186.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
Entry for principal kudu/127.23.186.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab.
I20260811 14:34:41.561885 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:40351
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:36513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:41.646659 14684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:41.646885 14684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:41.646910 14684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:41.648094 14684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:41.648144 14684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:41.648172 14684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:41.649817 14684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:36513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.195.keytab
--principal=kudu/127.23.186.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:40351
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:41.650084 14684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:41.650336 14684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:41.655638 14692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.655674 14695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.655638 14693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.655977 14684 server_base.cc:1034] running on GCE node
I20260811 14:34:41.656172 14684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:41.656381 14684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:41.657572 14684 hybrid_clock.cc:648] HybridClock initialized: now 1786458881657553 us; error 31 us; skew 500 ppm
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:41.660132 14684 init.cc:377] Logged in from keytab as kudu/127.23.186.195@KRBTEST.COM (short username kudu)
I20260811 14:34:41.660637 14684 webserver.cc:460] Webserver started at http://127.23.186.195:33467/ using document root <none> and password file <none>
I20260811 14:34:41.660887 14684 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:41.660938 14684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:41.661036 14684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:41.662179 14684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "793ffb4bc51d471383eba85b64fe2ebc"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "84b3583037b47d0bf3f564fe1a0be398"
server_key_iv: "8611147c3d10948797f4dc38ff121152"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.662612 14684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "793ffb4bc51d471383eba85b64fe2ebc"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "84b3583037b47d0bf3f564fe1a0be398"
server_key_iv: "8611147c3d10948797f4dc38ff121152"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.664175 14684 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:41.665048 14703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.665246 14684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:41.665341 14684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "793ffb4bc51d471383eba85b64fe2ebc"
format_stamp: "Formatted at 2026-08-11 14:34:41 on dist-test-slave-cbsf"
server_key: "84b3583037b47d0bf3f564fe1a0be398"
server_key_iv: "8611147c3d10948797f4dc38ff121152"
server_key_version: "encryptionkey@0"
I20260811 14:34:41.665445 14684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:41.678262 14684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:41.680279 14684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:41.680450 14684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:41.680742 14684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:41.681089 14684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:41.681125 14684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.681177 14684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:41.681205 14684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:41.687368 14684 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:37765
I20260811 14:34:41.687413 14830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:37765 every 8 connection(s)
I20260811 14:34:41.687748 14684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:41.688336 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14684
I20260811 14:34:41.688446 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:34:41.688680 24299 external_mini_cluster.cc:1428] Setting key ae99721a1d9e5721d9df4ed43021c9b2
Aug 11 14:34:41 dist-test-slave-cbsf krb5kdc[14255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458881, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.195@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:34:41.697451 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.688534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:42313 (local address 127.23.186.254:40351)
0811 14:34:41.688603 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.688606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.688872 (+ 266us) server_negotiation.cc:407] Connection header received
0811 14:34:41.689546 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.689547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.689560 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.689622 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:41.690661 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.691188 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.692212 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.692413 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.692496 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.694735 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:41.694736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:41.694737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:41.694745 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:41.695959 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.696391 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.696392 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.696393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.696454 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.696752 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.696753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.696754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.696875 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:41.696955 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.697266 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.697356 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 14:34:41.697641 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.688531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40351 (local address 127.23.186.195:42313)
0811 14:34:41.688728 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.688772 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.689440 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.689694 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.689698 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.690345 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:41.690556 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.690560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.691298 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.691299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.692119 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.692122 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.692506 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.692508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.692637 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.693187 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:34:41.693197 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:41.694632 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:41.696050 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:41.696058 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:41.696061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:41.696304 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:41.696526 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:34:41.696527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:34:41.696527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:34:41.696644 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:34:41.696946 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:41.696953 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:41.697113 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.697228 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.697396 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260811 14:34:41.698194 14832 heartbeater.cc:347] Connected to a master server at 127.23.186.254:40351
I20260811 14:34:41.698287 14832 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:41.698447 14832 heartbeater.cc:511] Master 127.23.186.254:40351 requested a full tablet report, sending...
I20260811 14:34:41.698776 14305 ts_manager.cc:194] Registered new tserver with Master: 793ffb4bc51d471383eba85b64fe2ebc (127.23.186.195:37765)
I20260811 14:34:41.699172 14305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.23.186.195@KRBTEST.COM'} at 127.23.186.195:42313
I20260811 14:34:41.700352 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:34:41.707600 14351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.701748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58466 (local address 127.23.186.254:40351)
0811 14:34:41.701809 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.701812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.701943 (+ 131us) server_negotiation.cc:407] Connection header received
0811 14:34:41.701987 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.701988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.701997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.702043 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:41.702539 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.703005 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.703714 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.703899 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.703956 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.705330 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:41.705331 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:41.705332 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:41.705338 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:41.706593 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.706915 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.706916 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.706916 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.706970 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:41.707134 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:41.707135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:41.707135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:41.707289 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:41.707363 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.707451 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.707511 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 14:34:41.709426 14305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58466:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:34:41.711133 14305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:34:41.717731 14844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.713457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:37765 (local address 127.0.0.1:41420)
0811 14:34:41.714158 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.714166 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.714217 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.714477 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.714478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.714480 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:41.714529 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.714530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.715770 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.715771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.716576 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.716579 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.717412 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.717414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.717497 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.717514 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.717589 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.717644 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260811 14:34:41.717758 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.713727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41420 (local address 127.23.186.195:37765)
0811 14:34:41.713891 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.713897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.714234 (+ 337us) server_negotiation.cc:407] Connection header received
0811 14:34:41.714283 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.714285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.714304 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.714383 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:41.714624 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.715661 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.716670 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.717320 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.717396 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.717452 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.717623 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.717664 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":93,"threads_started":1}
I20260811 14:34:41.718369 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.713457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43141 (local address 127.0.0.1:41336)
0811 14:34:41.714040 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.714049 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.714088 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.714578 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.714579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.714581 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:41.714638 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.714640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.715546 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.715547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.717211 (+ 1664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.717213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.718111 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.718112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.718182 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.718186 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.718236 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.718278 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":378,"thread_start_us":97,"threads_started":1}
I20260811 14:34:41.718472 14845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.713445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:43099 (local address 127.0.0.1:55686)
0811 14:34:41.713799 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.713848 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.713923 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.714147 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.714150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.714153 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:41.714243 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.714246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.715637 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.715637 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.716979 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.716982 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.718242 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.718243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.718292 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.718295 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.718337 (+ 42us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.718374 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":52,"thread_start_us":72,"threads_started":1}
I20260811 14:34:41.718762 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.713564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55686 (local address 127.23.186.193:43099)
0811 14:34:41.713961 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.713965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.713976 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:34:41.714018 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.714020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.714038 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.714136 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:41.714353 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.715543 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.717068 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.717776 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.718250 (+ 474us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.718314 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.718348 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.718410 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":117,"threads_started":1}
I20260811 14:34:41.718650 14758 tablet_service.cc:1467] Processing CreateTablet for tablet c88e721c1be04eb79a57edd632dc80cd (DEFAULT_TABLE table=test-table [id=ff0351efc3684574960b49c4081e89e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:41.718858 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.713751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41336 (local address 127.23.186.194:43141)
0811 14:34:41.714348 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.714353 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.714364 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:34:41.714405 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.714407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.714422 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.714495 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:41.714716 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.715439 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.717292 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.718005 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.718091 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.718143 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.718709 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.718762 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":89,"threads_started":1}
I20260811 14:34:41.719074 14758 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88e721c1be04eb79a57edd632dc80cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:41.719458 14603 tablet_service.cc:1467] Processing CreateTablet for tablet c88e721c1be04eb79a57edd632dc80cd (DEFAULT_TABLE table=test-table [id=ff0351efc3684574960b49c4081e89e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:41.719518 14449 tablet_service.cc:1467] Processing CreateTablet for tablet c88e721c1be04eb79a57edd632dc80cd (DEFAULT_TABLE table=test-table [id=ff0351efc3684574960b49c4081e89e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:41.719790 14603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88e721c1be04eb79a57edd632dc80cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:41.719838 14449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c88e721c1be04eb79a57edd632dc80cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:41.722630 14850 tablet_bootstrap.cc:492] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Bootstrap starting.
I20260811 14:34:41.723012 14851 tablet_bootstrap.cc:492] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Bootstrap starting.
I20260811 14:34:41.723402 14852 tablet_bootstrap.cc:492] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428: Bootstrap starting.
I20260811 14:34:41.723553 14850 tablet_bootstrap.cc:654] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:41.723742 14851 tablet_bootstrap.cc:654] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:41.724184 14850 log.cc:824] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:41.724244 14852 tablet_bootstrap.cc:654] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:41.724236 14851 log.cc:824] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:41.724848 14852 log.cc:824] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:41.725350 14850 tablet_bootstrap.cc:492] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: No bootstrap required, opened a new log
I20260811 14:34:41.725428 14850 ts_tablet_manager.cc:1397] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:34:41.725442 14851 tablet_bootstrap.cc:492] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: No bootstrap required, opened a new log
I20260811 14:34:41.725505 14851 ts_tablet_manager.cc:1397] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 14:34:41.725597 14852 tablet_bootstrap.cc:492] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428: No bootstrap required, opened a new log
I20260811 14:34:41.725668 14852 ts_tablet_manager.cc:1397] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 14:34:41.726933 14850 raft_consensus.cc:348] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.727124 14850 raft_consensus.cc:374] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:41.727173 14850 raft_consensus.cc:729] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4d7c7e1f7d043db93a1f19a06645f5f, State: Initialized, Role: FOLLOWER
I20260811 14:34:41.727344 14850 consensus_queue.cc:260] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.727659 14679 heartbeater.cc:503] Master 127.23.186.254:40351 was elected leader, sending a full tablet report...
I20260811 14:34:41.727551 14851 raft_consensus.cc:348] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.727661 14850 ts_tablet_manager.cc:1428] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:34:41.727735 14851 raft_consensus.cc:374] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:41.727643 14852 raft_consensus.cc:348] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.727771 14851 raft_consensus.cc:729] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 793ffb4bc51d471383eba85b64fe2ebc, State: Initialized, Role: FOLLOWER
I20260811 14:34:41.727787 14852 raft_consensus.cc:374] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:41.727826 14852 raft_consensus.cc:729] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc329def3c264d499f5332a4f48ba428, State: Initialized, Role: FOLLOWER
I20260811 14:34:41.727916 14851 consensus_queue.cc:260] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.727931 14852 consensus_queue.cc:260] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.728206 14851 ts_tablet_manager.cc:1428] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 14:34:41.728205 14852 ts_tablet_manager.cc:1428] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260811 14:34:41.728331 14525 heartbeater.cc:503] Master 127.23.186.254:40351 was elected leader, sending a full tablet report...
I20260811 14:34:41.728549 14832 heartbeater.cc:503] Master 127.23.186.254:40351 was elected leader, sending a full tablet report...
W20260811 14:34:41.776993 14680 tablet.cc:2369] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:34:41.808542 14859 raft_consensus.cc:482] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:41.808684 14859 raft_consensus.cc:504] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.809106 14859 leader_election.cc:288] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 793ffb4bc51d471383eba85b64fe2ebc (127.23.186.195:37765), d4d7c7e1f7d043db93a1f19a06645f5f (127.23.186.194:43141)
I20260811 14:34:41.812099 14848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.809391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:42241 (local address 127.23.186.195:37765)
0811 14:34:41.809442 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.809444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.809609 (+ 165us) server_negotiation.cc:407] Connection header received
0811 14:34:41.809671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.809673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.809684 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.809750 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:41.809916 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.810383 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.811328 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.811722 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.811801 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.811813 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.811969 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.812019 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:34:41.812114 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.809278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:37765 (local address 127.23.186.193:42241)
0811 14:34:41.809523 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.809533 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.809592 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.809779 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.809780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.809782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:41.809848 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.809850 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.810469 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.810470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.811224 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.811226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.811810 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.811811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.811894 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.811898 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.811941 (+ 43us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.811993 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":18,"thread_start_us":151,"threads_started":1}
I20260811 14:34:41.812145 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.809404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:52677 (local address 127.23.186.194:43141)
0811 14:34:41.809451 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.809453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.809461 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:34:41.809495 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.809496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.809506 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.809581 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:41.809781 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.810324 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.811343 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.811730 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.811818 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.811827 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.812020 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.812068 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:41.812126 14527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.809277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:43141 (local address 127.23.186.193:52677)
0811 14:34:41.809350 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.809365 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:41.809419 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:41.809611 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:41.809613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:41.809615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:41.809700 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.809702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.810414 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.810415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.811222 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:41.811224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.811834 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.811835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:41.811920 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.811923 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:41.811995 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:34:41.812037 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 14:34:41.812589 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88e721c1be04eb79a57edd632dc80cd" candidate_uuid: "cc329def3c264d499f5332a4f48ba428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793ffb4bc51d471383eba85b64fe2ebc" is_pre_election: true
I20260811 14:34:41.812808 14781 raft_consensus.cc:2454] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc329def3c264d499f5332a4f48ba428 in term 0.
I20260811 14:34:41.812971 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88e721c1be04eb79a57edd632dc80cd" candidate_uuid: "cc329def3c264d499f5332a4f48ba428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" is_pre_election: true
I20260811 14:34:41.813133 14628 raft_consensus.cc:2454] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc329def3c264d499f5332a4f48ba428 in term 0.
I20260811 14:34:41.813133 14401 leader_election.cc:302] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 793ffb4bc51d471383eba85b64fe2ebc, cc329def3c264d499f5332a4f48ba428; no voters:
I20260811 14:34:41.813280 14859 raft_consensus.cc:2781] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:34:41.813359 14859 raft_consensus.cc:482] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:34:41.813393 14859 raft_consensus.cc:3037] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:41.814167 14859 raft_consensus.cc:504] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.814329 14859 leader_election.cc:288] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [CANDIDATE]: Term 1 election: Requested vote from peers 793ffb4bc51d471383eba85b64fe2ebc (127.23.186.195:37765), d4d7c7e1f7d043db93a1f19a06645f5f (127.23.186.194:43141)
I20260811 14:34:41.814491 14781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88e721c1be04eb79a57edd632dc80cd" candidate_uuid: "cc329def3c264d499f5332a4f48ba428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "793ffb4bc51d471383eba85b64fe2ebc"
I20260811 14:34:41.814579 14781 raft_consensus.cc:3037] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:41.814538 14628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c88e721c1be04eb79a57edd632dc80cd" candidate_uuid: "cc329def3c264d499f5332a4f48ba428" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f"
I20260811 14:34:41.814651 14628 raft_consensus.cc:3037] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:41.815279 14781 raft_consensus.cc:2454] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc329def3c264d499f5332a4f48ba428 in term 1.
I20260811 14:34:41.815451 14401 leader_election.cc:302] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 793ffb4bc51d471383eba85b64fe2ebc, cc329def3c264d499f5332a4f48ba428; no voters:
I20260811 14:34:41.815486 14628 raft_consensus.cc:2454] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc329def3c264d499f5332a4f48ba428 in term 1.
I20260811 14:34:41.815554 14859 raft_consensus.cc:2781] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:41.815762 14859 raft_consensus.cc:686] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [term 1 LEADER]: Becoming Leader. State: Replica: cc329def3c264d499f5332a4f48ba428, State: Running, Role: LEADER
I20260811 14:34:41.816001 14859 consensus_queue.cc:237] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } }
I20260811 14:34:41.816982 14305 catalog_manager.cc:5495] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc329def3c264d499f5332a4f48ba428 (127.23.186.193). New cstate: current_term: 1 leader_uuid: "cc329def3c264d499f5332a4f48ba428" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc329def3c264d499f5332a4f48ba428" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 43099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 } health_report { overall_health: UNKNOWN } } }
I20260811 14:34:41.834496 14847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.831856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55688 (local address 127.23.186.193:43099)
0811 14:34:41.831928 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:41.831930 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:41.831940 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:34:41.831988 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:41.831989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:41.832027 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:41.832102 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:34:41.832277 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.832867 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.833663 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:41.833875 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:41.833959 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:41.834134 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:34:41.834262 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:34:41.834358 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:41.834376 (+ 18us) server_negotiation.cc:299] Negotiation successful
0811 14:34:41.834399 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:34:41.836464 14781 raft_consensus.cc:1261] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 1 FOLLOWER]: Refusing update from remote peer cc329def3c264d499f5332a4f48ba428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:41.836464 14628 raft_consensus.cc:1261] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 1 FOLLOWER]: Refusing update from remote peer cc329def3c264d499f5332a4f48ba428: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:41.836879 14859 consensus_queue.cc:1035] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "793ffb4bc51d471383eba85b64fe2ebc" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 37765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:41.837059 14863 consensus_queue.cc:1035] T c88e721c1be04eb79a57edd632dc80cd P cc329def3c264d499f5332a4f48ba428 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4d7c7e1f7d043db93a1f19a06645f5f" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 43141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:41.840154 14867 mvcc.cc:205] Tried to move back new op lower bound from 7317335579999174656 to 7317335579919069184. Current Snapshot: MvccSnapshot[applied={T|T < 7317335579999174656}]
I20260811 14:34:41.842446 14304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:41.842571 14304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58466:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:41.843379 14304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7d23578e99245c3aea4a52d00ed7481: Sending DeleteTablet for 3 replicas of tablet c88e721c1be04eb79a57edd632dc80cd
I20260811 14:34:41.843729 14449 tablet_service.cc:1514] Processing DeleteTablet for tablet c88e721c1be04eb79a57edd632dc80cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:41 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:55686
I20260811 14:34:41.843757 14758 tablet_service.cc:1514] Processing DeleteTablet for tablet c88e721c1be04eb79a57edd632dc80cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:41 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:41420
I20260811 14:34:41.843973 14603 tablet_service.cc:1514] Processing DeleteTablet for tablet c88e721c1be04eb79a57edd632dc80cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:41 UTC) from {username='kudu', principal='kudu/127.23.186.254@KRBTEST.COM'} at 127.0.0.1:41336
I20260811 14:34:41.844014 14880 tablet_replica.cc:330] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: stopping tablet replica
I20260811 14:34:41.844121 14880 raft_consensus.cc:2229] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:41.844221 14881 tablet_replica.cc:330] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: stopping tablet replica
I20260811 14:34:41.844245 14880 raft_consensus.cc:2258] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:41.844352 14881 raft_consensus.cc:2229] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:41.844460 14881 raft_consensus.cc:2258] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:41.844775 14880 ts_tablet_manager.cc:1905] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:41.844839 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14377
I20260811 14:34:41.844971 14881 ts_tablet_manager.cc:1905] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:41.846858 14881 ts_tablet_manager.cc:1918] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:41.846892 14880 ts_tablet_manager.cc:1918] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:41.846982 14880 log.cc:1196] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/c88e721c1be04eb79a57edd632dc80cd
I20260811 14:34:41.846982 14881 log.cc:1196] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/c88e721c1be04eb79a57edd632dc80cd
I20260811 14:34:41.847294 14880 ts_tablet_manager.cc:1939] T c88e721c1be04eb79a57edd632dc80cd P 793ffb4bc51d471383eba85b64fe2ebc: Deleting consensus metadata
I20260811 14:34:41.847317 14881 ts_tablet_manager.cc:1939] T c88e721c1be04eb79a57edd632dc80cd P d4d7c7e1f7d043db93a1f19a06645f5f: Deleting consensus metadata
I20260811 14:34:41.847887 14289 catalog_manager.cc:4841] TS 793ffb4bc51d471383eba85b64fe2ebc (127.23.186.195:37765): tablet c88e721c1be04eb79a57edd632dc80cd (table test-table [id=ff0351efc3684574960b49c4081e89e2]) successfully deleted
I20260811 14:34:41.848402 14288 catalog_manager.cc:4841] TS d4d7c7e1f7d043db93a1f19a06645f5f (127.23.186.194:43141): tablet c88e721c1be04eb79a57edd632dc80cd (table test-table [id=ff0351efc3684574960b49c4081e89e2]) successfully deleted
W20260811 14:34:41.853984 14287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:43099 (error 108)
I20260811 14:34:41.854164 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14532
I20260811 14:34:41.857478 14846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:41.857242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:43099 (local address 127.0.0.1:55704)
0811 14:34:41.857317 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:41.857382 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:43099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 14:34:41.857683 14287 catalog_manager.cc:4584] TS cc329def3c264d499f5332a4f48ba428 (127.23.186.193:43099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c88e721c1be04eb79a57edd632dc80cd: Network error: Client connection negotiation failed: client connection to 127.23.186.193:43099: connect: Connection refused (error 111)
I20260811 14:34:41.861223 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14684
I20260811 14:34:41.869685 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14270
2026-08-11T14:34:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2093 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T14:34:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:41Z Disabled control of system clock
I20260811 14:34:41.902549 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33473
--webserver_interface=127.23.186.254
--webserver_port=0
--builtin_ntp_servers=127.23.186.212:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:33473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:41.989411 14886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:41.989653 14886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:41.989675 14886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:41.990810 14886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:41.990849 14886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:41.990862 14886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:41.990875 14886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:41.990887 14886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:41.992365 14886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:33473
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:33473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:41.992610 14886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:41.992955 14886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:34:41.998014 14886 server_base.cc:1034] running on GCE node
W20260811 14:34:41.998020 14891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.997956 14892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:41.997967 14894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:41.998596 14886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:41.998917 14886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:42.000080 14886 hybrid_clock.cc:648] HybridClock initialized: now 1786458882000062 us; error 37 us; skew 500 ppm
I20260811 14:34:42.001458 14886 webserver.cc:460] Webserver started at http://127.23.186.254:34291/ using document root <none> and password file <none>
I20260811 14:34:42.001748 14886 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:42.001820 14886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:42.001976 14886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:42.003090 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "5e2402a6d96644299db24b04243b5687"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "e29b085b72fa928735a41efeed5e1044"
server_key_iv: "ce4ec67f3c7482cda4118938fe8578a7"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.003461 14886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "5e2402a6d96644299db24b04243b5687"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "e29b085b72fa928735a41efeed5e1044"
server_key_iv: "ce4ec67f3c7482cda4118938fe8578a7"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.004908 14886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:42.005650 14903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.005831 14886 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.005936 14886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "5e2402a6d96644299db24b04243b5687"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "e29b085b72fa928735a41efeed5e1044"
server_key_iv: "ce4ec67f3c7482cda4118938fe8578a7"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.006011 14886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:42.016005 14886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:42.016424 14886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:42.016585 14886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:42.021271 14886 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:33473
I20260811 14:34:42.021309 14965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:33473 every 8 connection(s)
I20260811 14:34:42.021629 14886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:42.022651 14966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:42.025204 14966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687: Bootstrap starting.
I20260811 14:34:42.025846 14966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:42.026160 14966 log.cc:824] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:42.026902 14966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687: No bootstrap required, opened a new log
I20260811 14:34:42.027940 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14886
I20260811 14:34:42.028059 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:42.028250 24299 external_mini_cluster.cc:1428] Setting key c8b1227158d0b8ad1f8e34d4c7743a6e
I20260811 14:34:42.029014 14966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2402a6d96644299db24b04243b5687" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33473 } }
I20260811 14:34:42.029239 14966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:42.029281 14966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e2402a6d96644299db24b04243b5687, State: Initialized, Role: FOLLOWER
I20260811 14:34:42.029414 14966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2402a6d96644299db24b04243b5687" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33473 } }
I20260811 14:34:42.029493 14966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:42.029541 14966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:42.029600 14966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:42.030475 14966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2402a6d96644299db24b04243b5687" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33473 } }
I20260811 14:34:42.030596 14966 leader_election.cc:302] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e2402a6d96644299db24b04243b5687; no voters:
I20260811 14:34:42.030777 14966 leader_election.cc:288] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:42.030840 14971 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:42.030990 14971 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [term 1 LEADER]: Becoming Leader. State: Replica: 5e2402a6d96644299db24b04243b5687, State: Running, Role: LEADER
I20260811 14:34:42.031061 14966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:42.031081 14971 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2402a6d96644299db24b04243b5687" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33473 } }
I20260811 14:34:42.031456 14972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e2402a6d96644299db24b04243b5687" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2402a6d96644299db24b04243b5687" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33473 } } }
I20260811 14:34:42.031445 14971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e2402a6d96644299db24b04243b5687. Latest consensus state: current_term: 1 leader_uuid: "5e2402a6d96644299db24b04243b5687" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e2402a6d96644299db24b04243b5687" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 33473 } } }
I20260811 14:34:42.031548 14972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:42.031554 14971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:42.031955 14977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:42.032544 14977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:42.034785 14977 catalog_manager.cc:1347] Generated new cluster ID: fe91ee5762454704bf41b240667b69e1
I20260811 14:34:42.034848 14977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:42.035753 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.028908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.23.186.254:33473)
0811 14:34:42.029176 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.029186 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.029222 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:34:42.029366 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.029373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.029452 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.029610 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:42.029841 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.032004 (+ 2163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.033224 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.033804 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.033926 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.034970 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:42.034975 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:42.034975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:42.035193 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:42.035238 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:42.035254 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:42.035335 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.035434 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.035526 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":117,"threads_started":1}
I20260811 14:34:42.049309 14977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:42.050099 14977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:42.060571 14977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687: Generated new TSK 0
I20260811 14:34:42.060851 14977 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:42.072162 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.193:0
--local_ip_for_outbound_sockets=127.23.186.193
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33473
--builtin_ntp_servers=127.23.186.212:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:42.159983 14990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:42.160257 14990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:42.160300 14990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:42.161597 14990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:42.161669 14990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:42.161716 14990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.193
I20260811 14:34:42.163163 14990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.23.186.193
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.193
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:42.163471 14990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:42.163842 14990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:42.169207 14996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:42.169275 14998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:42.169443 14990 server_base.cc:1034] running on GCE node
W20260811 14:34:42.169240 14995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:42.169731 14990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:42.170012 14990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:42.171197 14990 hybrid_clock.cc:648] HybridClock initialized: now 1786458882171182 us; error 28 us; skew 500 ppm
I20260811 14:34:42.172850 14990 webserver.cc:460] Webserver started at http://127.23.186.193:43393/ using document root <none> and password file <none>
I20260811 14:34:42.173161 14990 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:42.173218 14990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:42.173403 14990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:42.174710 14990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data/instance:
uuid: "8b44abac45ec4dbc8102e71839023e77"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "0f300455d2e86b98f67af4470c5037a7"
server_key_iv: "e3db667e5c7fbf392dfa8ac3f7c7bd3e"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.175140 14990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance:
uuid: "8b44abac45ec4dbc8102e71839023e77"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "0f300455d2e86b98f67af4470c5037a7"
server_key_iv: "e3db667e5c7fbf392dfa8ac3f7c7bd3e"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.176668 14990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260811 14:34:42.177608 15005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.177783 14990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:34:42.177909 14990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal
uuid: "8b44abac45ec4dbc8102e71839023e77"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "0f300455d2e86b98f67af4470c5037a7"
server_key_iv: "e3db667e5c7fbf392dfa8ac3f7c7bd3e"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.177994 14990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:42.201632 14990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:42.202052 14990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:42.202181 14990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:42.202399 14990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:42.202790 14990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:42.202827 14990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.202874 14990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:42.202916 14990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.209828 14990 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.193:44347
I20260811 14:34:42.209877 15148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.193:44347 every 8 connection(s)
I20260811 14:34:42.210343 14990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/data/info.pb
I20260811 14:34:42.215934 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.211152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.193:46269 (local address 127.23.186.254:33473)
0811 14:34:42.211223 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.211230 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.211490 (+ 260us) server_negotiation.cc:407] Connection header received
0811 14:34:42.212022 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.212023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.212036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.212103 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:42.212862 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.213367 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.214439 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.214634 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.214685 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.215688 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:42.215689 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:42.215690 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:42.215694 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:42.215710 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:42.215715 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:42.215767 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.215835 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.215866 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:34:42.216199 15151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.211127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33473 (local address 127.23.186.193:46269)
0811 14:34:42.211341 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.211388 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.211921 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.212161 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.212165 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.212173 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:42.212780 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.212785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.213468 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.213469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.214356 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.214358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.214712 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.214713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.214831 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.215524 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:42.215525 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:42.215556 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:42.215582 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:42.215767 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:42.215770 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:42.215772 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.215823 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.215921 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":106,"threads_started":1}
I20260811 14:34:42.216923 15149 heartbeater.cc:347] Connected to a master server at 127.23.186.254:33473
I20260811 14:34:42.217028 15149 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:42.217206 15149 heartbeater.cc:511] Master 127.23.186.254:33473 requested a full tablet report, sending...
I20260811 14:34:42.217752 14923 ts_manager.cc:194] Registered new tserver with Master: 8b44abac45ec4dbc8102e71839023e77 (127.23.186.193:44347)
I20260811 14:34:42.218468 14923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.193:46269
I20260811 14:34:42.219156 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 14990
I20260811 14:34:42.219264 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-0/wal/instance
I20260811 14:34:42.219468 24299 external_mini_cluster.cc:1428] Setting key 251a2e7ff8c241b2dc50de6d267a1d8d
I20260811 14:34:42.220407 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.194:0
--local_ip_for_outbound_sockets=127.23.186.194
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33473
--builtin_ntp_servers=127.23.186.212:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:42.307881 15152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:42.308149 15152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:42.308219 15152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:42.309621 15152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:42.309682 15152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:42.309715 15152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.194
I20260811 14:34:42.311410 15152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.23.186.194
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.194
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:42.311672 15152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:42.311971 15152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:42.317391 15157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:42.317379 15158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:42.317379 15160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:42.317639 15152 server_base.cc:1034] running on GCE node
I20260811 14:34:42.317866 15152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:42.318152 15152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:42.319303 15152 hybrid_clock.cc:648] HybridClock initialized: now 1786458882319283 us; error 37 us; skew 500 ppm
I20260811 14:34:42.320636 15152 webserver.cc:460] Webserver started at http://127.23.186.194:45521/ using document root <none> and password file <none>
I20260811 14:34:42.320923 15152 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:42.320979 15152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:42.321076 15152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:42.322192 15152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data/instance:
uuid: "feefe1ba27e741f995b51686d7ac99a0"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "6704456d5afce7abe9d088362bda25eb"
server_key_iv: "bbc7fba53c06bee176fb5f0dfdbb4d2f"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.322521 15152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance:
uuid: "feefe1ba27e741f995b51686d7ac99a0"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "6704456d5afce7abe9d088362bda25eb"
server_key_iv: "bbc7fba53c06bee176fb5f0dfdbb4d2f"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.323799 15152 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:42.324612 15167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.324834 15152 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:42.324966 15152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal
uuid: "feefe1ba27e741f995b51686d7ac99a0"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "6704456d5afce7abe9d088362bda25eb"
server_key_iv: "bbc7fba53c06bee176fb5f0dfdbb4d2f"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.325045 15152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:42.349664 15152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:42.350034 15152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:42.350193 15152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:42.350462 15152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:42.350827 15152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:42.350881 15152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.350924 15152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:42.350966 15152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.357282 15152 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.194:36755
I20260811 14:34:42.357334 15298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.194:36755 every 8 connection(s)
I20260811 14:34:42.357748 15152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/data/info.pb
I20260811 14:34:42.363497 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.358643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:56905 (local address 127.23.186.254:33473)
0811 14:34:42.358709 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.358711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.358981 (+ 270us) server_negotiation.cc:407] Connection header received
0811 14:34:42.359553 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.359555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.359569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.359621 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:42.360338 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.360945 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.361971 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.362173 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.362261 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.363176 (+ 915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:42.363177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:42.363178 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:42.363184 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:42.363205 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:42.363209 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:42.363271 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.363372 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.363408 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 14:34:42.363757 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.358624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33473 (local address 127.23.186.194:56905)
0811 14:34:42.358859 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.358896 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.359422 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.359697 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.359702 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.359710 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:42.360239 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.360243 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.361059 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.361060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.361873 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.361875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.362274 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.362275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.362422 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.363023 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:42.363023 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:42.363054 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:42.363081 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:42.363271 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:42.363279 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:42.363283 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.363362 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.363498 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":114,"threads_started":1}
I20260811 14:34:42.364339 15299 heartbeater.cc:347] Connected to a master server at 127.23.186.254:33473
I20260811 14:34:42.364454 15299 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:42.364634 15299 heartbeater.cc:511] Master 127.23.186.254:33473 requested a full tablet report, sending...
I20260811 14:34:42.365109 14923 ts_manager.cc:194] Registered new tserver with Master: feefe1ba27e741f995b51686d7ac99a0 (127.23.186.194:36755)
I20260811 14:34:42.365470 14923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.194:56905
I20260811 14:34:42.366986 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 15152
I20260811 14:34:42.367146 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal/instance
I20260811 14:34:42.367352 24299 external_mini_cluster.cc:1428] Setting key 4d2e6f4770d6cd81c3faa21c01f00fc1
I20260811 14:34:42.368302 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.23.186.195:0
--local_ip_for_outbound_sockets=127.23.186.195
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33473
--builtin_ntp_servers=127.23.186.212:40647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 14:34:42.455125 15302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:42.455364 15302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:42.455389 15302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:42.456635 15302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:42.456682 15302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:42.456794 15302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.23.186.195
I20260811 14:34:42.458226 15302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.23.186.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.23.186.195
--webserver_port=0
--tserver_master_addrs=127.23.186.254:33473
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.23.186.195
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:42.458515 15302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:42.458858 15302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:42.464169 15308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:42.464191 15310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:42.464310 15307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:42.464493 15302 server_base.cc:1034] running on GCE node
I20260811 14:34:42.464793 15302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:42.465056 15302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:42.466205 15302 hybrid_clock.cc:648] HybridClock initialized: now 1786458882466187 us; error 39 us; skew 500 ppm
I20260811 14:34:42.467643 15302 webserver.cc:460] Webserver started at http://127.23.186.195:46189/ using document root <none> and password file <none>
I20260811 14:34:42.467938 15302 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:42.467990 15302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:42.468165 15302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:42.469399 15302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data/instance:
uuid: "8605778ca9f24bcca977592dacedb75a"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "cb9245a6e4f41bf31497ef4ab927b406"
server_key_iv: "6a6f6e229cc306918114015092e06f09"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.469776 15302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance:
uuid: "8605778ca9f24bcca977592dacedb75a"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "cb9245a6e4f41bf31497ef4ab927b406"
server_key_iv: "6a6f6e229cc306918114015092e06f09"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.471303 15302 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:34:42.472133 15316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.472332 15302 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:42.472411 15302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal
uuid: "8605778ca9f24bcca977592dacedb75a"
format_stamp: "Formatted at 2026-08-11 14:34:42 on dist-test-slave-cbsf"
server_key: "cb9245a6e4f41bf31497ef4ab927b406"
server_key_iv: "6a6f6e229cc306918114015092e06f09"
server_key_version: "encryptionkey@0"
I20260811 14:34:42.472505 15302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:42.490024 15302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:42.490384 15302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:42.490536 15302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:42.490787 15302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 14:34:42.491150 15302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 14:34:42.491186 15302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.491228 15302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 14:34:42.491256 15302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:42.497648 15302 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.195:33977
I20260811 14:34:42.497694 15454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.195:33977 every 8 connection(s)
I20260811 14:34:42.498073 15302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/data/info.pb
I20260811 14:34:42.503765 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.498911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:59407 (local address 127.23.186.254:33473)
0811 14:34:42.498960 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.498962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.499255 (+ 293us) server_negotiation.cc:407] Connection header received
0811 14:34:42.499831 (+ 576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.499833 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.499846 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.499917 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:42.500682 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.501233 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.502300 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.502498 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.502575 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.503446 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:42.503448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:42.503448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:42.503454 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:42.503472 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:42.503481 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:42.503542 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.503639 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.503676 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:34:42.504019 15458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.498896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:33473 (local address 127.23.186.195:59407)
0811 14:34:42.499121 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.499161 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.499712 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.499998 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.500002 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.500011 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:34:42.500574 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.500578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.501357 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.501358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.502189 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.502192 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.502585 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.502586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.502732 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.503295 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 14:34:42.503295 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:34:42.503326 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 14:34:42.503352 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:34:42.503542 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:34:42.503549 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:34:42.503553 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.503641 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.503757 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260811 14:34:42.504577 15455 heartbeater.cc:347] Connected to a master server at 127.23.186.254:33473
I20260811 14:34:42.504770 15455 heartbeater.cc:464] Registering TS with master...
I20260811 14:34:42.504944 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 15302
I20260811 14:34:42.505005 15455 heartbeater.cc:511] Master 127.23.186.254:33473 requested a full tablet report, sending...
I20260811 14:34:42.505059 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal/instance
I20260811 14:34:42.505263 24299 external_mini_cluster.cc:1428] Setting key e1b86f8ccede31d93ebdc560930d9e2c
I20260811 14:34:42.505437 14923 ts_manager.cc:194] Registered new tserver with Master: 8605778ca9f24bcca977592dacedb75a (127.23.186.195:33977)
I20260811 14:34:42.505987 14923 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.23.186.195:59407
I20260811 14:34:42.506341 24299 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 14:34:42.511003 14970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.507916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46552 (local address 127.23.186.254:33473)
0811 14:34:42.507984 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.507987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.508101 (+ 114us) server_negotiation.cc:407] Connection header received
0811 14:34:42.508180 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.508181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.508194 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.508253 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:42.508452 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.509031 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.509799 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.509994 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.510069 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.510733 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:42.510734 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:42.510735 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:42.510740 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:42.510755 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:42.510759 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:42.510822 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.510884 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.510910 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:34:42.512539 14923 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46552:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 14:34:42.514230 14923 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 14:34:42.520519 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.516940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.23.186.193:44347)
0811 14:34:42.517148 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.517152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.517161 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:34:42.517202 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.517204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.517219 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.517291 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.517481 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.518609 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.519597 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.520099 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.520193 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.520294 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.520377 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.520416 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":105,"threads_started":1}
I20260811 14:34:42.520500 15468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.516634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44347 (local address 127.0.0.1:59090)
0811 14:34:42.516929 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.516938 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.516979 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.517358 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.517358 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.517360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.517404 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.517405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.518692 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.518693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.519504 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.519506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.520190 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.520191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.520269 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.520283 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.520362 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.520418 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":3,"thread_start_us":130,"threads_started":1}
I20260811 14:34:42.521190 15465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.516409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:36755 (local address 127.0.0.1:40536)
0811 14:34:42.516592 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.516634 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.516724 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.517447 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.517448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.517451 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.517500 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.517501 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.518438 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.518439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.519831 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.519834 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.520931 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.520931 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.520997 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.521002 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.521061 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.521105 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":66,"thread_start_us":84,"threads_started":1}
I20260811 14:34:42.521226 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.516571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.23.186.194:36755)
0811 14:34:42.517223 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.517228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.517237 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:34:42.517279 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.517281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.517296 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.517370 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.517581 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.518348 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.519928 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.520662 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.520811 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.520902 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.521084 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.521132 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":92,"threads_started":1}
I20260811 14:34:42.521454 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.516617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43062 (local address 127.23.186.195:33977)
0811 14:34:42.516909 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.516914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.516925 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:34:42.516972 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.516974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.516992 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.517083 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.517298 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.518493 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.520222 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.520993 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.521064 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.521155 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.521332 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.521374 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":151,"threads_started":1}
I20260811 14:34:42.521673 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.516423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:33977 (local address 127.0.0.1:43062)
0811 14:34:42.516789 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.516803 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.516859 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.517101 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.517104 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.517107 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.517198 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.517200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.518577 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.518577 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.520126 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.520128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.521191 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.521191 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.521255 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.521260 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.521317 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.521357 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":389,"thread_start_us":56,"threads_started":1}
I20260811 14:34:42.521660 15069 tablet_service.cc:1467] Processing CreateTablet for tablet 59ba3cd672264716866bf44ddc9cea2a (DEFAULT_TABLE table=test-table [id=5eb0b0674a9d4adda5ecab6bb813d47c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:42.522068 15069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59ba3cd672264716866bf44ddc9cea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:42.522300 15373 tablet_service.cc:1467] Processing CreateTablet for tablet 59ba3cd672264716866bf44ddc9cea2a (DEFAULT_TABLE table=test-table [id=5eb0b0674a9d4adda5ecab6bb813d47c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:42.522620 15373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59ba3cd672264716866bf44ddc9cea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:42.525512 15472 tablet_bootstrap.cc:492] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Bootstrap starting.
I20260811 14:34:42.525563 15471 tablet_bootstrap.cc:492] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77: Bootstrap starting.
I20260811 14:34:42.525519 15227 tablet_service.cc:1467] Processing CreateTablet for tablet 59ba3cd672264716866bf44ddc9cea2a (DEFAULT_TABLE table=test-table [id=5eb0b0674a9d4adda5ecab6bb813d47c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 14:34:42.525908 15227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 59ba3cd672264716866bf44ddc9cea2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:42.526208 15472 tablet_bootstrap.cc:654] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:42.526244 15471 tablet_bootstrap.cc:654] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:42.526515 15472 log.cc:824] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:42.526556 15471 log.cc:824] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:42.527323 15472 tablet_bootstrap.cc:492] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: No bootstrap required, opened a new log
I20260811 14:34:42.527331 15471 tablet_bootstrap.cc:492] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77: No bootstrap required, opened a new log
I20260811 14:34:42.527403 15472 ts_tablet_manager.cc:1397] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:34:42.527415 15471 ts_tablet_manager.cc:1397] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 14:34:42.529019 15475 tablet_bootstrap.cc:492] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Bootstrap starting.
I20260811 14:34:42.529706 15475 tablet_bootstrap.cc:654] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:42.530025 15475 log.cc:824] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:42.530176 15472 raft_consensus.cc:348] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.530176 15471 raft_consensus.cc:348] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.530360 15472 raft_consensus.cc:374] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:42.530360 15471 raft_consensus.cc:374] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:42.530400 15472 raft_consensus.cc:729] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8605778ca9f24bcca977592dacedb75a, State: Initialized, Role: FOLLOWER
I20260811 14:34:42.530409 15471 raft_consensus.cc:729] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b44abac45ec4dbc8102e71839023e77, State: Initialized, Role: FOLLOWER
I20260811 14:34:42.530539 15471 consensus_queue.cc:260] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.530622 15472 consensus_queue.cc:260] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.530838 15471 ts_tablet_manager.cc:1428] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:34:42.530921 15472 ts_tablet_manager.cc:1428] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260811 14:34:42.530993 15475 tablet_bootstrap.cc:492] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: No bootstrap required, opened a new log
I20260811 14:34:42.530993 15149 heartbeater.cc:503] Master 127.23.186.254:33473 was elected leader, sending a full tablet report...
I20260811 14:34:42.531059 15475 ts_tablet_manager.cc:1397] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 14:34:42.531713 15455 heartbeater.cc:503] Master 127.23.186.254:33473 was elected leader, sending a full tablet report...
I20260811 14:34:42.533703 15475 raft_consensus.cc:348] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.533879 15475 raft_consensus.cc:374] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:42.533926 15475 raft_consensus.cc:729] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feefe1ba27e741f995b51686d7ac99a0, State: Initialized, Role: FOLLOWER
I20260811 14:34:42.534053 15475 consensus_queue.cc:260] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.534340 15475 ts_tablet_manager.cc:1428] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260811 14:34:42.534411 15299 heartbeater.cc:503] Master 127.23.186.254:33473 was elected leader, sending a full tablet report...
W20260811 14:34:42.608420 15300 tablet.cc:2369] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 14:34:42.688619 15479 raft_consensus.cc:482] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:42.688818 15479 raft_consensus.cc:504] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.689244 15479 leader_election.cc:288] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8605778ca9f24bcca977592dacedb75a (127.23.186.195:33977), 8b44abac45ec4dbc8102e71839023e77 (127.23.186.193:44347)
I20260811 14:34:42.691300 15478 raft_consensus.cc:482] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 14:34:42.691425 15478 raft_consensus.cc:504] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.691790 15478 leader_election.cc:288] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers feefe1ba27e741f995b51686d7ac99a0 (127.23.186.194:36755), 8b44abac45ec4dbc8102e71839023e77 (127.23.186.193:44347)
I20260811 14:34:42.693171 15301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.689453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.195:33977 (local address 127.23.186.194:48345)
0811 14:34:42.689516 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.689530 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.689592 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.689751 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.689753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.689755 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.689841 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.689843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.690751 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.690752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.692076 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.692079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.692910 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.692911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.692987 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.692991 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.693041 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.693095 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":34}
I20260811 14:34:42.693231 15470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.689619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:51095 (local address 127.23.186.193:44347)
0811 14:34:42.689683 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.689686 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.689812 (+ 126us) server_negotiation.cc:407] Connection header received
0811 14:34:42.689854 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.689855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.689866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.689932 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.690131 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.690806 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.692200 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.692786 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.692843 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.692854 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.693098 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.693155 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:34:42.693401 15469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.689537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.194:48345 (local address 127.23.186.195:33977)
0811 14:34:42.689611 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.689613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.689628 (+ 15us) server_negotiation.cc:407] Connection header received
0811 14:34:42.689667 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.689672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.689682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.689742 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.689913 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.690659 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.692190 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.692806 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.692867 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.692873 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.693275 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.693324 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 14:34:42.693554 15480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.689479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44347 (local address 127.23.186.194:51095)
0811 14:34:42.689704 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.689714 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.689772 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.689951 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.689951 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.689953 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.690006 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.690007 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.690872 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.690873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.692076 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.692079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.692950 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.692951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.693013 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.693016 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.693425 (+ 409us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.693472 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":8,"thread_start_us":136,"threads_started":1}
I20260811 14:34:42.693688 15398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ba3cd672264716866bf44ddc9cea2a" candidate_uuid: "feefe1ba27e741f995b51686d7ac99a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8605778ca9f24bcca977592dacedb75a" is_pre_election: true
I20260811 14:34:42.693821 15398 raft_consensus.cc:2454] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feefe1ba27e741f995b51686d7ac99a0 in term 0.
I20260811 14:34:42.694149 15173 leader_election.cc:302] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8605778ca9f24bcca977592dacedb75a, feefe1ba27e741f995b51686d7ac99a0; no voters:
I20260811 14:34:42.694198 15093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ba3cd672264716866bf44ddc9cea2a" candidate_uuid: "feefe1ba27e741f995b51686d7ac99a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b44abac45ec4dbc8102e71839023e77" is_pre_election: true
I20260811 14:34:42.694334 15093 raft_consensus.cc:2454] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feefe1ba27e741f995b51686d7ac99a0 in term 0.
I20260811 14:34:42.694496 15479 raft_consensus.cc:2781] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 14:34:42.694566 15479 raft_consensus.cc:482] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 14:34:42.694593 15479 raft_consensus.cc:3037] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:42.695421 15479 raft_consensus.cc:504] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.695554 15458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.691971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.194:36755 (local address 127.23.186.195:34699)
0811 14:34:42.692065 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.692077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.692132 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.692342 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.692343 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.692344 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.692415 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.692417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.693344 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.693345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.694535 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.694537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.695293 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.695294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.695375 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.695379 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.695452 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.695496 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 14:34:42.695591 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.692022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:34699 (local address 127.23.186.194:36755)
0811 14:34:42.692210 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.692213 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.692221 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:34:42.692243 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.692244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.692254 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.692299 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.692491 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.693238 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.694625 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.695169 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.695246 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.695254 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.695472 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.695511 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260811 14:34:42.695716 15398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ba3cd672264716866bf44ddc9cea2a" candidate_uuid: "feefe1ba27e741f995b51686d7ac99a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8605778ca9f24bcca977592dacedb75a"
I20260811 14:34:42.695799 15398 raft_consensus.cc:3037] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:42.696413 15248 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ba3cd672264716866bf44ddc9cea2a" candidate_uuid: "8605778ca9f24bcca977592dacedb75a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feefe1ba27e741f995b51686d7ac99a0" is_pre_election: true
I20260811 14:34:42.696504 15398 raft_consensus.cc:2454] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feefe1ba27e741f995b51686d7ac99a0 in term 1.
I20260811 14:34:42.696539 15248 raft_consensus.cc:2379] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8605778ca9f24bcca977592dacedb75a in current term 1: Already voted for candidate feefe1ba27e741f995b51686d7ac99a0 in this term.
I20260811 14:34:42.696619 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.692088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44347 (local address 127.23.186.195:36893)
0811 14:34:42.692347 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.692357 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:34:42.692402 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:34:42.692781 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:34:42.692782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:34:42.692784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 14:34:42.692845 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.692847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.694404 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.694405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.695534 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:34:42.695537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.696295 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.696297 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:34:42.696377 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.696382 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 14:34:42.696443 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 14:34:42.696479 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":94,"threads_started":1}
I20260811 14:34:42.696744 15173 leader_election.cc:302] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8605778ca9f24bcca977592dacedb75a, feefe1ba27e741f995b51686d7ac99a0; no voters:
I20260811 14:34:42.696808 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.692499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.23.186.195:36893 (local address 127.23.186.193:44347)
0811 14:34:42.692627 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.692629 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.692636 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:34:42.692658 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.692659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.692667 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.692903 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 14:34:42.692934 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.694255 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.695637 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.696210 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.696295 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.696303 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.696476 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.696590 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":58,"threads_started":1}
I20260811 14:34:42.696942 15479 leader_election.cc:288] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [CANDIDATE]: Term 1 election: Requested vote from peers 8605778ca9f24bcca977592dacedb75a (127.23.186.195:33977), 8b44abac45ec4dbc8102e71839023e77 (127.23.186.193:44347)
I20260811 14:34:42.696966 15093 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ba3cd672264716866bf44ddc9cea2a" candidate_uuid: "feefe1ba27e741f995b51686d7ac99a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b44abac45ec4dbc8102e71839023e77"
I20260811 14:34:42.697008 15479 raft_consensus.cc:2781] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:42.697054 15479 raft_consensus.cc:686] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 1 LEADER]: Becoming Leader. State: Replica: feefe1ba27e741f995b51686d7ac99a0, State: Running, Role: LEADER
I20260811 14:34:42.697069 15092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "59ba3cd672264716866bf44ddc9cea2a" candidate_uuid: "8605778ca9f24bcca977592dacedb75a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b44abac45ec4dbc8102e71839023e77" is_pre_election: true
I20260811 14:34:42.697154 15479 consensus_queue.cc:237] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } }
I20260811 14:34:42.697058 15093 raft_consensus.cc:3037] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:42.697459 15318 leader_election.cc:302] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8605778ca9f24bcca977592dacedb75a; no voters: 8b44abac45ec4dbc8102e71839023e77, feefe1ba27e741f995b51686d7ac99a0
I20260811 14:34:42.697757 15478 raft_consensus.cc:2726] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 14:34:42.698097 15093 raft_consensus.cc:2454] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feefe1ba27e741f995b51686d7ac99a0 in term 1.
I20260811 14:34:42.698045 14923 catalog_manager.cc:5495] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to feefe1ba27e741f995b51686d7ac99a0 (127.23.186.194). New cstate: current_term: 1 leader_uuid: "feefe1ba27e741f995b51686d7ac99a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "feefe1ba27e741f995b51686d7ac99a0" member_type: VOTER last_known_addr { host: "127.23.186.194" port: 36755 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 } health_report { overall_health: UNKNOWN } } }
I20260811 14:34:42.705585 15466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.703172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40544 (local address 127.23.186.194:36755)
0811 14:34:42.703223 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:42.703226 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:42.703234 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:34:42.703274 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:42.703275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:42.703292 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:42.703367 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:34:42.703478 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.703956 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.704747 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:42.704946 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:42.705031 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:42.705195 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:34:42.705313 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:34:42.705401 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:42.705439 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 14:34:42.705462 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 14:34:42.707489 15093 raft_consensus.cc:1261] T 59ba3cd672264716866bf44ddc9cea2a P 8b44abac45ec4dbc8102e71839023e77 [term 1 FOLLOWER]: Refusing update from remote peer feefe1ba27e741f995b51686d7ac99a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:42.707508 15398 raft_consensus.cc:1261] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 1 FOLLOWER]: Refusing update from remote peer feefe1ba27e741f995b51686d7ac99a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:34:42.707803 15479 consensus_queue.cc:1035] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b44abac45ec4dbc8102e71839023e77" member_type: VOTER last_known_addr { host: "127.23.186.193" port: 44347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:42.707955 15484 consensus_queue.cc:1035] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8605778ca9f24bcca977592dacedb75a" member_type: VOTER last_known_addr { host: "127.23.186.195" port: 33977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:34:42.711210 15493 mvcc.cc:205] Tried to move back new op lower bound from 7317335583566835712 to 7317335583528144896. Current Snapshot: MvccSnapshot[applied={T|T < 7317335583566835712}]
I20260811 14:34:42.711795 15491 mvcc.cc:205] Tried to move back new op lower bound from 7317335583566835712 to 7317335583528144896. Current Snapshot: MvccSnapshot[applied={T|T < 7317335583566835712}]
I20260811 14:34:42.713095 14923 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:42.713186 14923 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46552:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 14:34:42.714026 14923 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5e2402a6d96644299db24b04243b5687: Sending DeleteTablet for 3 replicas of tablet 59ba3cd672264716866bf44ddc9cea2a
I20260811 14:34:42.714407 15069 tablet_service.cc:1514] Processing DeleteTablet for tablet 59ba3cd672264716866bf44ddc9cea2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:42 UTC) from {username='slave'} at 127.0.0.1:59090
I20260811 14:34:42.714406 15227 tablet_service.cc:1514] Processing DeleteTablet for tablet 59ba3cd672264716866bf44ddc9cea2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:42 UTC) from {username='slave'} at 127.0.0.1:40536
I20260811 14:34:42.714416 15373 tablet_service.cc:1514] Processing DeleteTablet for tablet 59ba3cd672264716866bf44ddc9cea2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 14:34:42 UTC) from {username='slave'} at 127.0.0.1:43062
I20260811 14:34:42.714785 15503 tablet_replica.cc:330] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: stopping tablet replica
I20260811 14:34:42.714900 15503 raft_consensus.cc:2229] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 14:34:42.715011 15503 raft_consensus.cc:2258] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:42.714999 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14990
I20260811 14:34:42.715173 15502 tablet_replica.cc:330] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: stopping tablet replica
I20260811 14:34:42.715263 15502 raft_consensus.cc:2229] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 14:34:42.715351 15502 raft_consensus.cc:2258] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 14:34:42.716988 15502 ts_tablet_manager.cc:1905] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:42.718792 15502 ts_tablet_manager.cc:1918] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:42.718879 15502 log.cc:1196] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-1/wal/wals/59ba3cd672264716866bf44ddc9cea2a
I20260811 14:34:42.719165 15502 ts_tablet_manager.cc:1939] T 59ba3cd672264716866bf44ddc9cea2a P feefe1ba27e741f995b51686d7ac99a0: Deleting consensus metadata
I20260811 14:34:42.719835 15503 ts_tablet_manager.cc:1905] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 14:34:42.721915 15503 ts_tablet_manager.cc:1918] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 14:34:42.722010 15503 log.cc:1196] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Deleting WAL directory at /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786458616937656-24299-0/minicluster-data/ts-2/wal/wals/59ba3cd672264716866bf44ddc9cea2a
I20260811 14:34:42.722245 15503 ts_tablet_manager.cc:1939] T 59ba3cd672264716866bf44ddc9cea2a P 8605778ca9f24bcca977592dacedb75a: Deleting consensus metadata
I20260811 14:34:42.722496 14905 catalog_manager.cc:4841] TS feefe1ba27e741f995b51686d7ac99a0 (127.23.186.194:36755): tablet 59ba3cd672264716866bf44ddc9cea2a (table test-table [id=5eb0b0674a9d4adda5ecab6bb813d47c]) successfully deleted
I20260811 14:34:42.723209 14907 catalog_manager.cc:4841] TS 8605778ca9f24bcca977592dacedb75a (127.23.186.195:33977): tablet 59ba3cd672264716866bf44ddc9cea2a (table test-table [id=5eb0b0674a9d4adda5ecab6bb813d47c]) successfully deleted
W20260811 14:34:42.728410 14905 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.23.186.193:44347 (error 108)
I20260811 14:34:42.728730 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 15152
I20260811 14:34:42.733925 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:42.732929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.193:44347 (local address 127.0.0.1:59106)
0811 14:34:42.733817 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 14:34:42.733882 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.193:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":859}
W20260811 14:34:42.734158 14905 catalog_manager.cc:4584] TS 8b44abac45ec4dbc8102e71839023e77 (127.23.186.193:44347): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 59ba3cd672264716866bf44ddc9cea2a: Network error: Client connection negotiation failed: client connection to 127.23.186.193:44347: connect: Connection refused (error 111)
I20260811 14:34:42.738730 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 15302
I20260811 14:34:42.747035 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 14886
2026-08-11T14:34:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (877 ms)
[----------] 6 tests from EncryptionPolicyTest (8984 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:42 dist-test-slave-cbsf krb5kdc[15508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:42 dist-test-slave-cbsf krb5kdc[15508](info): set up 2 sockets
Aug 11 14:34:42 dist-test-slave-cbsf krb5kdc[15508](info): commencing operation
krb5kdc: starting...
W20260811 14:34:43.868189 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:43 dist-test-slave-cbsf krb5kdc[15508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:43.999284 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34329
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38825
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34329
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:44.087816 15521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:44.088090 15521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:44.088141 15521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:44.089344 15521 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:44.089418 15521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:44.089444 15521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:44.089473 15521 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:44.089500 15521 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:44.091002 15521 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38825
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34329
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:44.091300 15521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:44.091538 15521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:34:44.096665 15521 server_base.cc:1034] running on GCE node
W20260811 14:34:44.096647 15527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:44.096624 15530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:44.096632 15528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:44.097143 15521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:44.097483 15521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:44.098647 15521 hybrid_clock.cc:648] HybridClock initialized: now 1786458884098641 us; error 36 us; skew 500 ppm
Aug 11 14:34:44 dist-test-slave-cbsf krb5kdc[15508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458884, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:44.106272 15521 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:44.106796 15521 webserver.cc:460] Webserver started at http://127.0.0.1:46773/ using document root <none> and password file <none>
I20260811 14:34:44.107029 15521 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:44.107105 15521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:44.107257 15521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:44.108387 15521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "b66565e40f854c12af19aad8c261b2d6"
format_stamp: "Formatted at 2026-08-11 14:34:44 on dist-test-slave-cbsf"
server_key: "745bd97dadd8c16c90ee7ee84e9722a9"
server_key_iv: "73a7a07316a6c0a98557b43f0bfaf27f"
server_key_version: "encryptionkey@0"
I20260811 14:34:44.108960 15521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "b66565e40f854c12af19aad8c261b2d6"
format_stamp: "Formatted at 2026-08-11 14:34:44 on dist-test-slave-cbsf"
server_key: "745bd97dadd8c16c90ee7ee84e9722a9"
server_key_iv: "73a7a07316a6c0a98557b43f0bfaf27f"
server_key_version: "encryptionkey@0"
I20260811 14:34:44.110507 15521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:44.111310 15537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:44.111482 15521 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:34:44.111593 15521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "b66565e40f854c12af19aad8c261b2d6"
format_stamp: "Formatted at 2026-08-11 14:34:44 on dist-test-slave-cbsf"
server_key: "745bd97dadd8c16c90ee7ee84e9722a9"
server_key_iv: "73a7a07316a6c0a98557b43f0bfaf27f"
server_key_version: "encryptionkey@0"
I20260811 14:34:44.111673 15521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:44.131211 15521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:44.133559 15521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:44.133741 15521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:44.138186 15521 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34329
I20260811 14:34:44.138228 15599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34329 every 8 connection(s)
I20260811 14:34:44.138605 15521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:44.139532 15600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:44.142076 15600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6: Bootstrap starting.
I20260811 14:34:44.142657 15600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:44.142918 15600 log.cc:824] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:44.143643 15600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6: No bootstrap required, opened a new log
I20260811 14:34:44.145401 15600 raft_consensus.cc:348] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66565e40f854c12af19aad8c261b2d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34329 } }
I20260811 14:34:44.145550 15600 raft_consensus.cc:374] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:44.145573 15600 raft_consensus.cc:729] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66565e40f854c12af19aad8c261b2d6, State: Initialized, Role: FOLLOWER
I20260811 14:34:44.145727 15600 consensus_queue.cc:260] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66565e40f854c12af19aad8c261b2d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34329 } }
I20260811 14:34:44.145826 15600 raft_consensus.cc:388] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:44.145860 15600 raft_consensus.cc:482] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:44.145851 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 15521
I20260811 14:34:44.145932 15600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:44.145973 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:44.146183 24299 external_mini_cluster.cc:1428] Setting key 5e71f35787f2eb46bac454c264bd0883
I20260811 14:34:44.146683 15600 raft_consensus.cc:504] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66565e40f854c12af19aad8c261b2d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34329 } }
I20260811 14:34:44.146816 15600 leader_election.cc:302] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b66565e40f854c12af19aad8c261b2d6; no voters:
I20260811 14:34:44.146996 15600 leader_election.cc:288] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:44.147038 15606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:44.147179 15606 raft_consensus.cc:686] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [term 1 LEADER]: Becoming Leader. State: Replica: b66565e40f854c12af19aad8c261b2d6, State: Running, Role: LEADER
I20260811 14:34:44.147289 15600 sys_catalog.cc:564] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:44.147284 15606 consensus_queue.cc:237] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66565e40f854c12af19aad8c261b2d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34329 } }
I20260811 14:34:44.148744 15610 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b66565e40f854c12af19aad8c261b2d6. Latest consensus state: current_term: 1 leader_uuid: "b66565e40f854c12af19aad8c261b2d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66565e40f854c12af19aad8c261b2d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34329 } } }
I20260811 14:34:44.148854 15610 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:44.149120 15608 sys_catalog.cc:455] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b66565e40f854c12af19aad8c261b2d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b66565e40f854c12af19aad8c261b2d6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34329 } } }
I20260811 14:34:44.149235 15608 sys_catalog.cc:458] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6 [sys.catalog]: This master's current role is: LEADER
W20260811 14:34:44.149600 15623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:34:44.149667 15623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:34:44.149717 15624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:44.149873 15624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:44.151358 15624 catalog_manager.cc:1347] Generated new cluster ID: 7922d522b661479d969dd2499896a5da
I20260811 14:34:44.151409 15624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:34:44 dist-test-slave-cbsf krb5kdc[15508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:34:44.158511 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:44.146750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43467 (local address 127.0.0.1:34329)
0811 14:34:44.146994 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:44.147005 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:44.147048 (+ 43us) server_negotiation.cc:407] Connection header received
0811 14:34:44.147191 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:44.147199 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:44.147309 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:44.147488 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:44.148289 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:44.149842 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:44.150596 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:44.151203 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:44.151316 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:44.154339 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:44.154344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:44.154345 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:44.154358 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:44.156444 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:44.157065 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:44.157068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:44.157069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:44.157209 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:44.157491 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:44.157493 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:44.157493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:44.157833 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:44.157992 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:44.158177 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 14:34:44.158294 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":106,"threads_started":1}
I20260811 14:34:44.159541 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:44.171414 15624 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:44.172190 15624 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:44.175859 15624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b66565e40f854c12af19aad8c261b2d6: Generated new TSK 0
I20260811 14:34:44.176064 15624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:44.177333 15605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:44.168849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60005 (local address 127.0.0.1:34329)
0811 14:34:44.168899 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:44.168901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:44.169139 (+ 238us) server_negotiation.cc:407] Connection header received
0811 14:34:44.169201 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:44.169202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:44.169215 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:44.169278 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:44.169974 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:44.171231 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:44.172355 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:44.172684 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:44.172796 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:44.174334 (+ 1538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:44.174336 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:44.174336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:44.174345 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:44.176064 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:44.176520 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:44.176521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:44.176521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:44.176578 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:44.176839 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:44.176839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:44.176840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:44.176940 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:44.177020 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:44.177194 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 14:34:44.177252 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260811 14:34:44.179617 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 15521
W20260811 14:34:44.190214 15506 connection.cc:535] client connection to 127.0.0.1:34329 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34329): Connection reset by peer (error 104)
2026-08-11T14:34:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:44 dist-test-slave-cbsf krb5kdc[15636](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:44 dist-test-slave-cbsf krb5kdc[15636](info): set up 2 sockets
Aug 11 14:34:44 dist-test-slave-cbsf krb5kdc[15636](info): commencing operation
krb5kdc: starting...
W20260811 14:34:45.313783 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:45 dist-test-slave-cbsf krb5kdc[15636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:45.445802 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32957
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32957
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:45.533813 15650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:45.534051 15650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:45.534078 15650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:45.535313 15650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:45.535363 15650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:45.535377 15650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:45.535390 15650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:45.535449 15650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:45.537048 15650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32957
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:45.537360 15650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:45.537639 15650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:45.542950 15657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:45.543051 15650 server_base.cc:1034] running on GCE node
W20260811 14:34:45.542946 15658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:45.542950 15661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:45.543365 15650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:45.543725 15650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:45.544914 15650 hybrid_clock.cc:648] HybridClock initialized: now 1786458885544889 us; error 42 us; skew 500 ppm
Aug 11 14:34:45 dist-test-slave-cbsf krb5kdc[15636](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458885, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:45.547909 15650 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:45.548480 15650 webserver.cc:460] Webserver started at http://127.0.0.1:32839/ using document root <none> and password file <none>
I20260811 14:34:45.548820 15650 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:45.548897 15650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:45.549058 15650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:45.550228 15650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "441c6ebb837c4edea78dc873ab77fa33"
format_stamp: "Formatted at 2026-08-11 14:34:45 on dist-test-slave-cbsf"
server_key: "2d1eb0139f2dd710be08a54549e06249"
server_key_iv: "c69a241af45ec7fd908da7c8e6a7eb21"
server_key_version: "encryptionkey@0"
I20260811 14:34:45.550628 15650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "441c6ebb837c4edea78dc873ab77fa33"
format_stamp: "Formatted at 2026-08-11 14:34:45 on dist-test-slave-cbsf"
server_key: "2d1eb0139f2dd710be08a54549e06249"
server_key_iv: "c69a241af45ec7fd908da7c8e6a7eb21"
server_key_version: "encryptionkey@0"
I20260811 14:34:45.552208 15650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:45.553128 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:45.553334 15650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260811 14:34:45.553447 15650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "441c6ebb837c4edea78dc873ab77fa33"
format_stamp: "Formatted at 2026-08-11 14:34:45 on dist-test-slave-cbsf"
server_key: "2d1eb0139f2dd710be08a54549e06249"
server_key_iv: "c69a241af45ec7fd908da7c8e6a7eb21"
server_key_version: "encryptionkey@0"
I20260811 14:34:45.553532 15650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:45.575194 15650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:45.577777 15650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:45.577950 15650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:45.582351 15650 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32957
I20260811 14:34:45.582412 15727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32957 every 8 connection(s)
I20260811 14:34:45.582810 15650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:45.583766 15728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:45.586325 15728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33: Bootstrap starting.
I20260811 14:34:45.586969 15728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:45.587273 15728 log.cc:824] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:45.588045 15728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33: No bootstrap required, opened a new log
I20260811 14:34:45.589807 15728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441c6ebb837c4edea78dc873ab77fa33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32957 } }
I20260811 14:34:45.589968 15728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:45.589993 15728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 441c6ebb837c4edea78dc873ab77fa33, State: Initialized, Role: FOLLOWER
I20260811 14:34:45.590205 15728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441c6ebb837c4edea78dc873ab77fa33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32957 } }
I20260811 14:34:45.590299 15728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:45.590325 15728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:45.590371 15728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:45.590978 15728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441c6ebb837c4edea78dc873ab77fa33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32957 } }
I20260811 14:34:45.591090 15728 leader_election.cc:302] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 441c6ebb837c4edea78dc873ab77fa33; no voters:
I20260811 14:34:45.591306 15728 leader_election.cc:288] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:45.591358 15733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:45.591583 15733 raft_consensus.cc:686] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [term 1 LEADER]: Becoming Leader. State: Replica: 441c6ebb837c4edea78dc873ab77fa33, State: Running, Role: LEADER
I20260811 14:34:45.591606 15728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:45.591722 15733 consensus_queue.cc:237] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441c6ebb837c4edea78dc873ab77fa33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32957 } }
I20260811 14:34:45.592176 15738 sys_catalog.cc:455] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 441c6ebb837c4edea78dc873ab77fa33. Latest consensus state: current_term: 1 leader_uuid: "441c6ebb837c4edea78dc873ab77fa33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441c6ebb837c4edea78dc873ab77fa33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32957 } } }
I20260811 14:34:45.592269 15738 sys_catalog.cc:458] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:45.592422 15736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "441c6ebb837c4edea78dc873ab77fa33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "441c6ebb837c4edea78dc873ab77fa33" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32957 } } }
I20260811 14:34:45.592566 15736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:45.592516 15744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:45.592855 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 15650
I20260811 14:34:45.592969 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:45.593201 24299 external_mini_cluster.cc:1428] Setting key 07349a39b507fd3a94228f6f63ca4863
I20260811 14:34:45.593411 15744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:45.595235 15744 catalog_manager.cc:1347] Generated new cluster ID: 00bb91213f1947a7a64bdcc8eeb57b5b
I20260811 14:34:45.595306 15744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:45.601783 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:45.593734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43335 (local address 127.0.0.1:32957)
0811 14:34:45.594546 (+ 812us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:45.594556 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:45.594592 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:34:45.594744 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:45.594752 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:45.594815 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:45.594964 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:45.596762 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:45.598209 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:45.599439 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:45.600075 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:45.600185 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:45.601042 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:45.601047 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:45.601048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:45.601274 (+ 226us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:45.601315 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:45.601330 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:45.601405 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:45.601533 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 14:34:45.601565 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":89,"threads_started":1}
I20260811 14:34:45.602818 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:45.605621 15744 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:45.606295 15744 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:45.609032 15755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:45.605009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32863 (local address 127.0.0.1:32957)
0811 14:34:45.605059 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:45.605062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:45.605256 (+ 194us) server_negotiation.cc:407] Connection header received
0811 14:34:45.605311 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:45.605312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:45.605325 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:45.605389 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:45.605617 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:45.606354 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:45.607394 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:45.607738 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:45.607816 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:45.608574 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:45.608575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:45.608576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:45.608582 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:45.608599 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:45.608604 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:45.608673 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:45.608932 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 14:34:45.608947 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:45.609228 15744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 441c6ebb837c4edea78dc873ab77fa33: Generated new TSK 0
I20260811 14:34:45.609385 15744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:45.612995 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 15650
W20260811 14:34:45.622599 15634 connection.cc:535] client connection to 127.0.0.1:32957 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32957): Connection reset by peer (error 104)
2026-08-11T14:34:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:45 dist-test-slave-cbsf krb5kdc[15765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:45 dist-test-slave-cbsf krb5kdc[15765](info): set up 2 sockets
Aug 11 14:34:45 dist-test-slave-cbsf krb5kdc[15765](info): commencing operation
krb5kdc: starting...
W20260811 14:34:46.741503 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:46 dist-test-slave-cbsf krb5kdc[15765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:46.876299 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36145
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:46.964380 15778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:46.964617 15778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:46.964645 15778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:46.965893 15778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:46.965947 15778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:46.965962 15778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:46.965975 15778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:46.965987 15778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:46.967514 15778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36145
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:46.967739 15778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:46.967937 15778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:46.973009 15786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:46.973029 15778 server_base.cc:1034] running on GCE node
W20260811 14:34:46.972981 15787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:46.972999 15789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:46.973399 15778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:46.973753 15778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:46.974920 15778 hybrid_clock.cc:648] HybridClock initialized: now 1786458886974909 us; error 37 us; skew 500 ppm
Aug 11 14:34:46 dist-test-slave-cbsf krb5kdc[15765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458886, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:46.977669 15778 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:46.978181 15778 webserver.cc:460] Webserver started at http://127.0.0.1:40483/ using document root <none> and password file <none>
I20260811 14:34:46.978446 15778 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:46.978516 15778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:46.978669 15778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:46.979779 15778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "7857108996a74933a55bb6022249eb2a"
format_stamp: "Formatted at 2026-08-11 14:34:46 on dist-test-slave-cbsf"
server_key: "923e7de12265c9bca655f62c7dfed88f"
server_key_iv: "bdd20c834059448bac26f6264ecab74a"
server_key_version: "encryptionkey@0"
I20260811 14:34:46.980154 15778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "7857108996a74933a55bb6022249eb2a"
format_stamp: "Formatted at 2026-08-11 14:34:46 on dist-test-slave-cbsf"
server_key: "923e7de12265c9bca655f62c7dfed88f"
server_key_iv: "bdd20c834059448bac26f6264ecab74a"
server_key_version: "encryptionkey@0"
I20260811 14:34:46.981691 15778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:34:46.982514 15799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:46.982728 15778 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:46.982853 15778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "7857108996a74933a55bb6022249eb2a"
format_stamp: "Formatted at 2026-08-11 14:34:46 on dist-test-slave-cbsf"
server_key: "923e7de12265c9bca655f62c7dfed88f"
server_key_iv: "bdd20c834059448bac26f6264ecab74a"
server_key_version: "encryptionkey@0"
I20260811 14:34:46.982944 15778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:46.986994 15778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:46.989476 15778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:46.989670 15778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:46.994355 15778 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36145
I20260811 14:34:46.994436 15860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36145 every 8 connection(s)
I20260811 14:34:46.994871 15778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:46.996299 15861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:46.999534 15861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a: Bootstrap starting.
I20260811 14:34:47.000340 15861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:47.000610 15861 log.cc:824] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:47.001389 15861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a: No bootstrap required, opened a new log
I20260811 14:34:47.002624 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 15778
I20260811 14:34:47.002753 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:47.002941 24299 external_mini_cluster.cc:1428] Setting key b81457cb084fe3968c7fdc0657d4f2a5
I20260811 14:34:47.003204 15861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7857108996a74933a55bb6022249eb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260811 14:34:47.003372 15861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:47.003424 15861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7857108996a74933a55bb6022249eb2a, State: Initialized, Role: FOLLOWER
I20260811 14:34:47.003546 15861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7857108996a74933a55bb6022249eb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260811 14:34:47.003644 15861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:47.003679 15861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:47.003727 15861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:47.005016 15861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7857108996a74933a55bb6022249eb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260811 14:34:47.005167 15861 leader_election.cc:302] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7857108996a74933a55bb6022249eb2a; no voters:
I20260811 14:34:47.005376 15861 leader_election.cc:288] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:47.005399 15867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:47.005553 15867 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [term 1 LEADER]: Becoming Leader. State: Replica: 7857108996a74933a55bb6022249eb2a, State: Running, Role: LEADER
I20260811 14:34:47.005663 15867 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7857108996a74933a55bb6022249eb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } }
I20260811 14:34:47.005705 15861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:47.005870 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:47.003512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53563 (local address 127.0.0.1:36145)
0811 14:34:47.003745 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:47.003756 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:47.003921 (+ 165us) server_negotiation.cc:407] Connection header received
0811 14:34:47.004079 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:47.004086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:47.004136 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:47.004252 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:47.005138 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:47.005142 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:47.005142 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:47.005148 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:47.005201 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:47.005216 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:47.005296 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:47.005488 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 14:34:47.005539 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":116,"threads_started":1}
I20260811 14:34:47.005996 15867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7857108996a74933a55bb6022249eb2a. Latest consensus state: current_term: 1 leader_uuid: "7857108996a74933a55bb6022249eb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7857108996a74933a55bb6022249eb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } } }
I20260811 14:34:47.006064 15867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:47.006160 15868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7857108996a74933a55bb6022249eb2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7857108996a74933a55bb6022249eb2a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36145 } } }
I20260811 14:34:47.006278 15873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:47.006305 15868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:47.006834 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:47.006996 15873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:47.009282 15873 catalog_manager.cc:1347] Generated new cluster ID: 64f1361a21ef4ac58761703bea0e711f
I20260811 14:34:47.009348 15873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:47.009897 15866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:47.008487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38237 (local address 127.0.0.1:36145)
0811 14:34:47.008528 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:47.008530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:47.008641 (+ 111us) server_negotiation.cc:407] Connection header received
0811 14:34:47.008774 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:47.008775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:47.008782 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:47.008812 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:47.009659 (+ 847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:47.009660 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:47.009660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:47.009665 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:47.009682 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:47.009687 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:47.009748 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:47.009823 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 14:34:47.009836 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:34:47.013621 15873 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:47.013823 15873 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:47.020498 15873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7857108996a74933a55bb6022249eb2a: Generated new TSK 0
I20260811 14:34:47.020750 15873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:47.024503 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 15778
2026-08-11T14:34:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:47 dist-test-slave-cbsf krb5kdc[15897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:47 dist-test-slave-cbsf krb5kdc[15897](info): set up 2 sockets
Aug 11 14:34:47 dist-test-slave-cbsf krb5kdc[15897](info): commencing operation
krb5kdc: starting...
W20260811 14:34:48.161566 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:48 dist-test-slave-cbsf krb5kdc[15897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:48.295270 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:48.381567 15913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:48.381846 15913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:48.381889 15913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:48.383121 15913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:48.383198 15913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:48.383224 15913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:48.383255 15913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:48.383283 15913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:48.384889 15913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:48.385193 15913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:48.385434 15913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:48.390592 15920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:48.390658 15922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:48.390599 15919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:48.390672 15913 server_base.cc:1034] running on GCE node
I20260811 14:34:48.391063 15913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:48.391372 15913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:48.392544 15913 hybrid_clock.cc:648] HybridClock initialized: now 1786458888392526 us; error 44 us; skew 500 ppm
Aug 11 14:34:48 dist-test-slave-cbsf krb5kdc[15897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458888, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:48.395080 15913 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:48.395520 15913 webserver.cc:460] Webserver started at http://127.0.0.1:37579/ using document root <none> and password file <none>
I20260811 14:34:48.395751 15913 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:48.395817 15913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:48.395967 15913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:48.397166 15913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "206845533186417cb7baec5461330443"
format_stamp: "Formatted at 2026-08-11 14:34:48 on dist-test-slave-cbsf"
server_key: "503cd653df6284d5f8bbfb66e3ad02aa"
server_key_iv: "811991746fbc77441ce7bf78f988d423"
server_key_version: "encryptionkey@0"
I20260811 14:34:48.397526 15913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "206845533186417cb7baec5461330443"
format_stamp: "Formatted at 2026-08-11 14:34:48 on dist-test-slave-cbsf"
server_key: "503cd653df6284d5f8bbfb66e3ad02aa"
server_key_iv: "811991746fbc77441ce7bf78f988d423"
server_key_version: "encryptionkey@0"
I20260811 14:34:48.398916 15913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:34:48.399645 15929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:48.399823 15913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260811 14:34:48.399930 15913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "206845533186417cb7baec5461330443"
format_stamp: "Formatted at 2026-08-11 14:34:48 on dist-test-slave-cbsf"
server_key: "503cd653df6284d5f8bbfb66e3ad02aa"
server_key_iv: "811991746fbc77441ce7bf78f988d423"
server_key_version: "encryptionkey@0"
I20260811 14:34:48.400005 15913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:48.422547 15913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:48.424518 15913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:48.424731 15913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:48.428714 15913 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43859
I20260811 14:34:48.428781 15990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43859 every 8 connection(s)
I20260811 14:34:48.429162 15913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:48.430102 15991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:48.431705 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 15913
I20260811 14:34:48.431849 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:48.432035 24299 external_mini_cluster.cc:1428] Setting key 7a16fc79f548aeffd291d14cc9872880
I20260811 14:34:48.432987 15991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443: Bootstrap starting.
I20260811 14:34:48.433753 15991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:48.434067 15991 log.cc:824] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:48.434850 15991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443: No bootstrap required, opened a new log
I20260811 14:34:48.436771 15991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206845533186417cb7baec5461330443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 14:34:48.436960 15991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:48.437011 15991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 206845533186417cb7baec5461330443, State: Initialized, Role: FOLLOWER
I20260811 14:34:48.437183 15991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206845533186417cb7baec5461330443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 14:34:48.437278 15991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:48.437310 15991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:48.437366 15991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:48.438019 15991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206845533186417cb7baec5461330443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 14:34:48.438115 15991 leader_election.cc:302] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 206845533186417cb7baec5461330443; no voters:
I20260811 14:34:48.438315 15991 leader_election.cc:288] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:48.438365 15996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:48.438541 15996 raft_consensus.cc:686] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [term 1 LEADER]: Becoming Leader. State: Replica: 206845533186417cb7baec5461330443, State: Running, Role: LEADER
I20260811 14:34:48.438614 15991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:48.438654 15996 consensus_queue.cc:237] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206845533186417cb7baec5461330443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 14:34:48.439009 15996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 206845533186417cb7baec5461330443. Latest consensus state: current_term: 1 leader_uuid: "206845533186417cb7baec5461330443" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206845533186417cb7baec5461330443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260811 14:34:48.439108 15996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:48.439018 15997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "206845533186417cb7baec5461330443" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "206845533186417cb7baec5461330443" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
Aug 11 14:34:48 dist-test-slave-cbsf krb5kdc[15897](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:34:48.439288 15997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:48.439342 16005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:48.439888 16005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:48.442169 16005 catalog_manager.cc:1347] Generated new cluster ID: 480eef382e064f86a81da979d8d0376b
I20260811 14:34:48.442229 16005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:48.444195 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:48.432657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53047 (local address 127.0.0.1:43859)
0811 14:34:48.432865 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:48.432876 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:48.432911 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:34:48.433522 (+ 611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:48.433532 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:48.433595 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:48.433755 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:48.434476 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:48.435490 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:48.436259 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:48.436687 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:48.436863 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:48.439843 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:48.439848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:48.439848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:48.439864 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:48.442190 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:48.442630 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:48.442634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:48.442634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:48.442731 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:48.442910 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:48.442911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:48.442912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:48.443197 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:48.443339 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:48.443840 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 14:34:48.443981 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":105,"threads_started":1}
I20260811 14:34:48.445159 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:48.451520 16005 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:48.452075 16005 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:48.454363 16005 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 206845533186417cb7baec5461330443: Generated new TSK 0
I20260811 14:34:48.454571 16005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:48.459784 15994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:48.449007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35465 (local address 127.0.0.1:43859)
0811 14:34:48.449053 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:48.449055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:48.451089 (+ 2034us) server_negotiation.cc:407] Connection header received
0811 14:34:48.451232 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:48.451233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:48.451249 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:48.451328 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:48.453394 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:48.454185 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:48.455292 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:48.455515 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:48.455579 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:48.457115 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:48.457128 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:48.457128 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:48.457135 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:48.458552 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:48.458963 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:48.458964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:48.458964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:48.459026 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:48.459248 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:48.459249 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:48.459249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:48.459389 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:48.459488 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:48.459630 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 14:34:48.459721 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:48.461855 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 15913
2026-08-11T14:34:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:48 dist-test-slave-cbsf krb5kdc[16030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:48 dist-test-slave-cbsf krb5kdc[16030](info): set up 2 sockets
Aug 11 14:34:48 dist-test-slave-cbsf krb5kdc[16030](info): commencing operation
krb5kdc: starting...
W20260811 14:34:49.585044 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:49 dist-test-slave-cbsf krb5kdc[16030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:49.714133 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40601
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:49.799391 16046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:49.799675 16046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:49.799715 16046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:49.800938 16046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:49.801023 16046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:49.801046 16046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:49.801069 16046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:49.801110 16046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:49.802691 16046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:49.803014 16046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:49.803272 16046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:49.808324 16057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:49.808382 16054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:49.808341 16053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:49.808583 16046 server_base.cc:1034] running on GCE node
I20260811 14:34:49.808910 16046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:49.809217 16046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:49.810411 16046 hybrid_clock.cc:648] HybridClock initialized: now 1786458889810394 us; error 44 us; skew 500 ppm
Aug 11 14:34:49 dist-test-slave-cbsf krb5kdc[16030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458889, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:49.819159 16046 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:49.819599 16046 webserver.cc:460] Webserver started at http://127.0.0.1:42589/ using document root <none> and password file <none>
I20260811 14:34:49.819813 16046 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:49.819857 16046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:49.819989 16046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:49.821149 16046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "6f71301b94fc482bacac1e1d6541e4a5"
format_stamp: "Formatted at 2026-08-11 14:34:49 on dist-test-slave-cbsf"
server_key: "6d07bf6c995882baa3e5af1cc42b3610"
server_key_iv: "11e693b765a1266468f89fefc5a57dfc"
server_key_version: "encryptionkey@0"
I20260811 14:34:49.821458 16046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "6f71301b94fc482bacac1e1d6541e4a5"
format_stamp: "Formatted at 2026-08-11 14:34:49 on dist-test-slave-cbsf"
server_key: "6d07bf6c995882baa3e5af1cc42b3610"
server_key_iv: "11e693b765a1266468f89fefc5a57dfc"
server_key_version: "encryptionkey@0"
I20260811 14:34:49.822901 16046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 14:34:49.823691 16065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:49.823904 16046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:49.824016 16046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "6f71301b94fc482bacac1e1d6541e4a5"
format_stamp: "Formatted at 2026-08-11 14:34:49 on dist-test-slave-cbsf"
server_key: "6d07bf6c995882baa3e5af1cc42b3610"
server_key_iv: "11e693b765a1266468f89fefc5a57dfc"
server_key_version: "encryptionkey@0"
I20260811 14:34:49.824100 16046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:49.836117 16046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:49.838385 16046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:49.838562 16046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:49.842953 16046 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40601
I20260811 14:34:49.842984 16129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40601 every 8 connection(s)
I20260811 14:34:49.843375 16046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:49.844280 16131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:49.846750 16131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5: Bootstrap starting.
I20260811 14:34:49.847379 16131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:49.847679 16131 log.cc:824] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:49.848485 16131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5: No bootstrap required, opened a new log
I20260811 14:34:49.850183 16131 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f71301b94fc482bacac1e1d6541e4a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40601 } }
I20260811 14:34:49.850329 16131 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:49.850353 16131 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f71301b94fc482bacac1e1d6541e4a5, State: Initialized, Role: FOLLOWER
I20260811 14:34:49.850517 16131 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f71301b94fc482bacac1e1d6541e4a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40601 } }
I20260811 14:34:49.850611 16131 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:49.850597 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16046
I20260811 14:34:49.850648 16131 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:49.850710 16131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:49.850728 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:49.850945 24299 external_mini_cluster.cc:1428] Setting key 472d9546b372a89089cf8536ee011c3a
I20260811 14:34:49.851444 16131 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f71301b94fc482bacac1e1d6541e4a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40601 } }
I20260811 14:34:49.851553 16131 leader_election.cc:302] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f71301b94fc482bacac1e1d6541e4a5; no voters:
I20260811 14:34:49.851729 16131 leader_election.cc:288] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:49.851806 16137 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:49.851995 16131 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:49.852000 16137 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [term 1 LEADER]: Becoming Leader. State: Replica: 6f71301b94fc482bacac1e1d6541e4a5, State: Running, Role: LEADER
I20260811 14:34:49.852185 16137 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f71301b94fc482bacac1e1d6541e4a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40601 } }
I20260811 14:34:49.852576 16138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f71301b94fc482bacac1e1d6541e4a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f71301b94fc482bacac1e1d6541e4a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40601 } } }
I20260811 14:34:49.852720 16138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:49.852574 16139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f71301b94fc482bacac1e1d6541e4a5. Latest consensus state: current_term: 1 leader_uuid: "6f71301b94fc482bacac1e1d6541e4a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f71301b94fc482bacac1e1d6541e4a5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40601 } } }
I20260811 14:34:49.852871 16139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5 [sys.catalog]: This master's current role is: LEADER
W20260811 14:34:49.854439 16154 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:34:49.854508 16154 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:34:49.854561 16142 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:49.854719 16142 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:49.856355 16142 catalog_manager.cc:1347] Generated new cluster ID: 0b956d4fd6a94407a3e1a70ea9cd0772
I20260811 14:34:49.856423 16142 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:49.857728 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:49.851517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34743 (local address 127.0.0.1:40601)
0811 14:34:49.851732 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:49.851742 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:49.851778 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:34:49.851905 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:49.851912 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:49.851967 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:49.852403 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:49.852557 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:49.854160 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:49.855324 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:49.855950 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:49.856102 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:49.856963 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:49.856967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:49.856967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:49.857212 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:49.857252 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:49.857267 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:49.857340 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:49.857420 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 14:34:49.857491 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":163,"thread_start_us":98,"threads_started":1}
I20260811 14:34:49.858729 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:49.864142 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:49.860363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49733 (local address 127.0.0.1:40601)
0811 14:34:49.860429 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:49.860431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:49.860481 (+ 50us) server_negotiation.cc:407] Connection header received
0811 14:34:49.860560 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:49.860561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:49.860572 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:49.860653 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:49.860827 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:49.861570 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:49.862584 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:49.862894 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:49.862954 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:49.863737 (+ 783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:49.863739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:49.863739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:49.863745 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:49.863762 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:49.863766 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:49.863813 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:49.864030 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 14:34:49.864056 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 14:34:49.879669 16142 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:49.880277 16142 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:49.884335 16142 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f71301b94fc482bacac1e1d6541e4a5: Generated new TSK 0
I20260811 14:34:49.884572 16142 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:49.891006 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16046
2026-08-11T14:34:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:49 dist-test-slave-cbsf krb5kdc[16165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:49 dist-test-slave-cbsf krb5kdc[16165](info): set up 2 sockets
Aug 11 14:34:49 dist-test-slave-cbsf krb5kdc[16165](info): commencing operation
krb5kdc: starting...
W20260811 14:34:51.025525 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:51 dist-test-slave-cbsf krb5kdc[16165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458891, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:51.155169 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37903
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:51.240906 16180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:51.241142 16180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:51.241168 16180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:51.242313 16180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:51.242355 16180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:51.242369 16180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:51.242383 16180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:51.242394 16180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:51.243913 16180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37903
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:51.244227 16180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:51.244611 16180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:51.250232 16190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:51.250249 16186 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:51.250771 16187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:51.250759 16180 server_base.cc:1034] running on GCE node
I20260811 14:34:51.251044 16180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:51.251431 16180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:51.252562 16180 hybrid_clock.cc:648] HybridClock initialized: now 1786458891252533 us; error 49 us; skew 500 ppm
Aug 11 14:34:51 dist-test-slave-cbsf krb5kdc[16165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458891, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:51.256203 16180 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:51.256883 16180 webserver.cc:460] Webserver started at http://127.0.0.1:33967/ using document root <none> and password file <none>
I20260811 14:34:51.257189 16180 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:51.257268 16180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:51.257454 16180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:51.258837 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "c691aa382c6e4f2bb051308105dd5168"
format_stamp: "Formatted at 2026-08-11 14:34:51 on dist-test-slave-cbsf"
server_key: "95562977f2ea587dc9d0b4c5c1410ae3"
server_key_iv: "0ede16063c76bb33512d4b86d0e367a9"
server_key_version: "encryptionkey@0"
I20260811 14:34:51.259420 16180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "c691aa382c6e4f2bb051308105dd5168"
format_stamp: "Formatted at 2026-08-11 14:34:51 on dist-test-slave-cbsf"
server_key: "95562977f2ea587dc9d0b4c5c1410ae3"
server_key_iv: "0ede16063c76bb33512d4b86d0e367a9"
server_key_version: "encryptionkey@0"
I20260811 14:34:51.261524 16180 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260811 14:34:51.262602 16198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:51.262812 16180 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:51.262940 16180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "c691aa382c6e4f2bb051308105dd5168"
format_stamp: "Formatted at 2026-08-11 14:34:51 on dist-test-slave-cbsf"
server_key: "95562977f2ea587dc9d0b4c5c1410ae3"
server_key_iv: "0ede16063c76bb33512d4b86d0e367a9"
server_key_version: "encryptionkey@0"
I20260811 14:34:51.263005 16180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:51.267307 16180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:51.269771 16180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:51.269896 16180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:51.274173 16180 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37903
I20260811 14:34:51.274252 16257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37903 every 8 connection(s)
I20260811 14:34:51.274611 16180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:51.275734 16258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:51.278347 16258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168: Bootstrap starting.
I20260811 14:34:51.278998 16258 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:51.279311 16258 log.cc:824] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:51.280061 16258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168: No bootstrap required, opened a new log
I20260811 14:34:51.281445 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16180
I20260811 14:34:51.281594 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:51.281796 24299 external_mini_cluster.cc:1428] Setting key bf7c035dd8c07257e3fa9eefeb6b20c9
I20260811 14:34:51.281981 16258 raft_consensus.cc:348] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c691aa382c6e4f2bb051308105dd5168" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37903 } }
I20260811 14:34:51.282207 16258 raft_consensus.cc:374] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:51.282260 16258 raft_consensus.cc:729] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c691aa382c6e4f2bb051308105dd5168, State: Initialized, Role: FOLLOWER
I20260811 14:34:51.282445 16258 consensus_queue.cc:260] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c691aa382c6e4f2bb051308105dd5168" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37903 } }
I20260811 14:34:51.282555 16258 raft_consensus.cc:388] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:51.282596 16258 raft_consensus.cc:482] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:51.282644 16258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:51.283984 16258 raft_consensus.cc:504] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c691aa382c6e4f2bb051308105dd5168" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37903 } }
I20260811 14:34:51.284127 16258 leader_election.cc:302] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c691aa382c6e4f2bb051308105dd5168; no voters:
I20260811 14:34:51.284299 16258 leader_election.cc:288] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:51.284482 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:51.282439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33719 (local address 127.0.0.1:37903)
0811 14:34:51.282780 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:51.282798 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:51.282852 (+ 54us) server_negotiation.cc:407] Connection header received
0811 14:34:51.283002 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:51.283010 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:51.283050 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:51.283216 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:51.283949 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:51.283953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:51.283953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:51.283958 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:51.284004 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:51.284016 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:51.284103 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:51.284174 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 14:34:51.284220 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":111,"threads_started":1}
I20260811 14:34:51.284497 16266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:51.284629 16266 raft_consensus.cc:686] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [term 1 LEADER]: Becoming Leader. State: Replica: c691aa382c6e4f2bb051308105dd5168, State: Running, Role: LEADER
I20260811 14:34:51.284632 16258 sys_catalog.cc:564] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:51.285432 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:51.284768 16266 consensus_queue.cc:237] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c691aa382c6e4f2bb051308105dd5168" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37903 } }
I20260811 14:34:51.286718 16265 sys_catalog.cc:455] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c691aa382c6e4f2bb051308105dd5168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c691aa382c6e4f2bb051308105dd5168" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37903 } } }
I20260811 14:34:51.286882 16267 sys_catalog.cc:455] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c691aa382c6e4f2bb051308105dd5168. Latest consensus state: current_term: 1 leader_uuid: "c691aa382c6e4f2bb051308105dd5168" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c691aa382c6e4f2bb051308105dd5168" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37903 } } }
I20260811 14:34:51.287024 16265 sys_catalog.cc:458] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:51.287031 16267 sys_catalog.cc:458] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:51.288278 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:51.286922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59011 (local address 127.0.0.1:37903)
0811 14:34:51.286962 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:51.286964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:51.287174 (+ 210us) server_negotiation.cc:407] Connection header received
0811 14:34:51.287245 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:51.287246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:51.287253 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:51.287303 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:51.288042 (+ 739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:51.288043 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:51.288044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:51.288048 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:51.288065 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:51.288070 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:51.288117 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:51.288172 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 14:34:51.288187 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 14:34:51.288184 16285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:34:51.288429 16285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:34:51.288483 16287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:51.288632 16287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:51.290226 16287 catalog_manager.cc:1347] Generated new cluster ID: ab8c21b3007d45d1b30c6be48b5d87b5
I20260811 14:34:51.290285 16287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:51.299176 16287 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:51.299393 16287 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:51.308517 16287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c691aa382c6e4f2bb051308105dd5168: Generated new TSK 0
I20260811 14:34:51.308805 16287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:51.314206 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16180
2026-08-11T14:34:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:51 dist-test-slave-cbsf krb5kdc[16295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:51 dist-test-slave-cbsf krb5kdc[16295](info): set up 2 sockets
Aug 11 14:34:51 dist-test-slave-cbsf krb5kdc[16295](info): commencing operation
krb5kdc: starting...
W20260811 14:34:52.445541 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:52 dist-test-slave-cbsf krb5kdc[16295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:52.577445 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44385
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:52.666031 16310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:52.666270 16310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:52.666296 16310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:52.667423 16310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:52.667475 16310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:52.667490 16310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:52.667503 16310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:52.667515 16310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:52.669045 16310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:52.669265 16310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:52.669454 16310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:52.674639 16315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:52.674779 16310 server_base.cc:1034] running on GCE node
W20260811 14:34:52.674823 16318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:52.674669 16316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:52.675112 16310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:52.675432 16310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:52.676630 16310 hybrid_clock.cc:648] HybridClock initialized: now 1786458892676621 us; error 37 us; skew 500 ppm
Aug 11 14:34:52 dist-test-slave-cbsf krb5kdc[16295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458892, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:52.679317 16310 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:52.679806 16310 webserver.cc:460] Webserver started at http://127.0.0.1:39359/ using document root <none> and password file <none>
I20260811 14:34:52.680078 16310 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:52.680150 16310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:52.680306 16310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:52.681515 16310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "4e4a8f189b6346a2ba8d058cb0820e28"
format_stamp: "Formatted at 2026-08-11 14:34:52 on dist-test-slave-cbsf"
server_key: "1c6c4a18bc02acb9014057c07fddbcf2"
server_key_iv: "d9f22616435eda8d1bd6d16752f5d5f5"
server_key_version: "encryptionkey@0"
I20260811 14:34:52.681895 16310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "4e4a8f189b6346a2ba8d058cb0820e28"
format_stamp: "Formatted at 2026-08-11 14:34:52 on dist-test-slave-cbsf"
server_key: "1c6c4a18bc02acb9014057c07fddbcf2"
server_key_iv: "d9f22616435eda8d1bd6d16752f5d5f5"
server_key_version: "encryptionkey@0"
I20260811 14:34:52.683434 16310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:34:52.684201 16326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:52.684399 16310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:34:52.684540 16310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "4e4a8f189b6346a2ba8d058cb0820e28"
format_stamp: "Formatted at 2026-08-11 14:34:52 on dist-test-slave-cbsf"
server_key: "1c6c4a18bc02acb9014057c07fddbcf2"
server_key_iv: "d9f22616435eda8d1bd6d16752f5d5f5"
server_key_version: "encryptionkey@0"
I20260811 14:34:52.684620 16310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:52.709950 16310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:52.712628 16310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:52.712834 16310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:52.717192 16310 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44385
I20260811 14:34:52.717288 16394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44385 every 8 connection(s)
I20260811 14:34:52.717664 16310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:52.718683 16395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:52.721046 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16310
I20260811 14:34:52.721189 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:52.721212 16395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28: Bootstrap starting.
I20260811 14:34:52.721410 24299 external_mini_cluster.cc:1428] Setting key 36466032962886932b6a7dea55f796d8
I20260811 14:34:52.721917 16395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:52.722517 16395 log.cc:824] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:52.723356 16395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28: No bootstrap required, opened a new log
I20260811 14:34:52.725845 16395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260811 14:34:52.726002 16395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:52.726027 16395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e4a8f189b6346a2ba8d058cb0820e28, State: Initialized, Role: FOLLOWER
I20260811 14:34:52.726181 16395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260811 14:34:52.726245 16395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:52.726289 16395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:52.726341 16395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:52.727157 16395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260811 14:34:52.727277 16395 leader_election.cc:302] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e4a8f189b6346a2ba8d058cb0820e28; no voters:
I20260811 14:34:52.727500 16395 leader_election.cc:288] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:52.727564 16403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:52.727751 16403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [term 1 LEADER]: Becoming Leader. State: Replica: 4e4a8f189b6346a2ba8d058cb0820e28, State: Running, Role: LEADER
I20260811 14:34:52.727824 16395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:52.727850 16403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } }
I20260811 14:34:52.728314 16406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e4a8f189b6346a2ba8d058cb0820e28. Latest consensus state: current_term: 1 leader_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } } }
I20260811 14:34:52.728474 16406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:52.728767 16411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:52.729285 16404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e4a8f189b6346a2ba8d058cb0820e28" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44385 } } }
I20260811 14:34:52.729334 16411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:52.729453 16404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28 [sys.catalog]: This master's current role is: LEADER
Aug 11 14:34:52 dist-test-slave-cbsf krb5kdc[16295](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:34:52.731642 16411 catalog_manager.cc:1347] Generated new cluster ID: 5349c57257ee47c388f1b6b18f6c9970
I20260811 14:34:52.731705 16411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:52.734215 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:52.722080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:51883 (local address 127.0.0.1:44385)
0811 14:34:52.722265 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:52.722277 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:52.722319 (+ 42us) server_negotiation.cc:407] Connection header received
0811 14:34:52.722945 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:52.722953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:52.723055 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:52.723214 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:52.723946 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:52.725467 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:52.726383 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:52.727015 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:52.727138 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:52.730443 (+ 3305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:52.730448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:52.730448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:52.730463 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:52.732451 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:52.732989 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:52.732993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:52.732993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:52.733089 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:52.733329 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:52.733330 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:52.733330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:52.733582 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:52.733764 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:52.733881 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 14:34:52.734002 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260811 14:34:52.735574 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:52.743745 16411 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:52.744496 16411 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:52.747601 16411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e4a8f189b6346a2ba8d058cb0820e28: Generated new TSK 0
I20260811 14:34:52.747797 16411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:52.748292 16401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:52.737040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:45451 (local address 127.0.0.1:44385)
0811 14:34:52.737103 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:52.737105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:52.740918 (+ 3813us) server_negotiation.cc:407] Connection header received
0811 14:34:52.741008 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:52.741009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:52.741020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:52.741071 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:52.741655 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:52.742448 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:52.743236 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:52.743595 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:52.743675 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:52.745099 (+ 1424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:52.745101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:52.745101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:52.745108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:52.746574 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:52.746966 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:52.746967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:52.746968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:52.747035 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:52.747302 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:52.747303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:52.747303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:52.747458 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:52.747567 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:52.748173 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 14:34:52.748232 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":238,"server-negotiator.queue_time_us":40}
I20260811 14:34:52.750479 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16310
W20260811 14:34:52.759682 16292 connection.cc:535] client connection to 127.0.0.1:44385 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44385): Connection reset by peer (error 104)
2026-08-11T14:34:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:52 dist-test-slave-cbsf krb5kdc[16434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:52 dist-test-slave-cbsf krb5kdc[16434](info): set up 2 sockets
Aug 11 14:34:52 dist-test-slave-cbsf krb5kdc[16434](info): commencing operation
krb5kdc: starting...
W20260811 14:34:53.867918 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:53 dist-test-slave-cbsf krb5kdc[16434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458893, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:53.999951 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:54.085618 16449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:54.085829 16449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:54.085853 16449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:54.086933 16449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:54.086971 16449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:54.086987 16449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:54.086999 16449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:54.087011 16449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:54.088428 16449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:54.088631 16449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:54.088887 16449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:54.093856 16454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:54.094118 16449 server_base.cc:1034] running on GCE node
W20260811 14:34:54.093856 16458 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:54.093878 16455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:54.101989 16449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:54.102396 16449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:54.103610 16449 hybrid_clock.cc:648] HybridClock initialized: now 1786458894103588 us; error 52 us; skew 500 ppm
Aug 11 14:34:54 dist-test-slave-cbsf krb5kdc[16434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458894, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:54.106467 16449 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:54.106997 16449 webserver.cc:460] Webserver started at http://127.0.0.1:41691/ using document root <none> and password file <none>
I20260811 14:34:54.107203 16449 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:54.107246 16449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:54.107344 16449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:54.108435 16449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "99d80fa128564112aceefcdadb319a87"
format_stamp: "Formatted at 2026-08-11 14:34:54 on dist-test-slave-cbsf"
server_key: "12a0ceee7baaca9dca74a457cf29fd21"
server_key_iv: "03220b76961ea4257991f78150028d89"
server_key_version: "encryptionkey@0"
I20260811 14:34:54.108829 16449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "99d80fa128564112aceefcdadb319a87"
format_stamp: "Formatted at 2026-08-11 14:34:54 on dist-test-slave-cbsf"
server_key: "12a0ceee7baaca9dca74a457cf29fd21"
server_key_iv: "03220b76961ea4257991f78150028d89"
server_key_version: "encryptionkey@0"
I20260811 14:34:54.110229 16449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:54.111166 16466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:54.111393 16449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:54.111501 16449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "99d80fa128564112aceefcdadb319a87"
format_stamp: "Formatted at 2026-08-11 14:34:54 on dist-test-slave-cbsf"
server_key: "12a0ceee7baaca9dca74a457cf29fd21"
server_key_iv: "03220b76961ea4257991f78150028d89"
server_key_version: "encryptionkey@0"
I20260811 14:34:54.111579 16449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:54.120658 16449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:54.123163 16449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:54.123344 16449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:54.127936 16449 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40293
I20260811 14:34:54.127983 16531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40293 every 8 connection(s)
I20260811 14:34:54.128412 16449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:54.129436 16532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:54.131842 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87: Bootstrap starting.
I20260811 14:34:54.132472 16532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:54.132820 16532 log.cc:824] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:54.133618 16532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87: No bootstrap required, opened a new log
I20260811 14:34:54.135216 16532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d80fa128564112aceefcdadb319a87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40293 } }
I20260811 14:34:54.135360 16532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:54.135408 16532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99d80fa128564112aceefcdadb319a87, State: Initialized, Role: FOLLOWER
I20260811 14:34:54.135588 16532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d80fa128564112aceefcdadb319a87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40293 } }
I20260811 14:34:54.135654 16532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:54.135710 16532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:54.135761 16532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:54.136317 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16449
I20260811 14:34:54.136329 16532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d80fa128564112aceefcdadb319a87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40293 } }
I20260811 14:34:54.136413 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:54.136444 16532 leader_election.cc:302] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99d80fa128564112aceefcdadb319a87; no voters:
I20260811 14:34:54.136623 16532 leader_election.cc:288] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:54.136672 24299 external_mini_cluster.cc:1428] Setting key 388ae4c45180e0b7e05e8e7de503d70b
I20260811 14:34:54.136722 16536 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:54.136904 16536 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [term 1 LEADER]: Becoming Leader. State: Replica: 99d80fa128564112aceefcdadb319a87, State: Running, Role: LEADER
I20260811 14:34:54.136917 16532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:54.137005 16536 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d80fa128564112aceefcdadb319a87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40293 } }
I20260811 14:34:54.137698 16538 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99d80fa128564112aceefcdadb319a87. Latest consensus state: current_term: 1 leader_uuid: "99d80fa128564112aceefcdadb319a87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d80fa128564112aceefcdadb319a87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40293 } } }
I20260811 14:34:54.137712 16537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99d80fa128564112aceefcdadb319a87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99d80fa128564112aceefcdadb319a87" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40293 } } }
I20260811 14:34:54.137805 16538 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:54.137811 16537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:54.138026 16547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:54.138554 16547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:54.140305 16547 catalog_manager.cc:1347] Generated new cluster ID: 891e44898945430daa03cfe703dcef05
I20260811 14:34:54.140447 16547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:54.142915 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:54.137445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:44999 (local address 127.0.0.1:40293)
0811 14:34:54.137647 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:54.137659 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:54.137754 (+ 95us) server_negotiation.cc:407] Connection header received
0811 14:34:54.137997 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:54.138006 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:54.138150 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:54.138327 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:54.138592 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:54.139651 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:54.140880 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:54.141324 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:54.141451 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:54.142232 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:54.142236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:54.142236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:54.142433 (+ 197us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:54.142465 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:54.142477 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:54.142580 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:54.142663 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 14:34:54.142692 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":80,"threads_started":1}
I20260811 14:34:54.143949 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:54.148862 16543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:54.145884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:57355 (local address 127.0.0.1:40293)
0811 14:34:54.145939 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:54.145941 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:54.145948 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:34:54.145969 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:54.145969 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:54.145996 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:54.146066 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:54.146255 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:54.146766 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:54.147492 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:54.147729 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:54.147809 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:54.148538 (+ 729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:54.148539 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:54.148539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:54.148544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:54.148562 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:54.148567 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:54.148644 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:54.148742 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 14:34:54.148756 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 14:34:54.170473 16547 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:54.171255 16547 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:54.182590 16547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99d80fa128564112aceefcdadb319a87: Generated new TSK 0
I20260811 14:34:54.182861 16547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:54.190760 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16449
W20260811 14:34:54.199622 16432 connection.cc:535] client connection to 127.0.0.1:40293 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40293): Connection reset by peer (error 104)
2026-08-11T14:34:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:54 dist-test-slave-cbsf krb5kdc[16567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:54 dist-test-slave-cbsf krb5kdc[16567](info): set up 2 sockets
Aug 11 14:34:54 dist-test-slave-cbsf krb5kdc[16567](info): commencing operation
krb5kdc: starting...
W20260811 14:34:55.308897 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:55 dist-test-slave-cbsf krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458895, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:55.443241 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38667
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:55.527951 16581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:55.528227 16581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:55.528270 16581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:55.529479 16581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:55.529554 16581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:55.529580 16581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:55.529606 16581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:55.529639 16581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:55.531155 16581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:55.531461 16581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:55.531702 16581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:55.536878 16587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:55.536933 16581 server_base.cc:1034] running on GCE node
W20260811 14:34:55.536866 16586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:55.536864 16589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:55.537441 16581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:55.537776 16581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:55.538978 16581 hybrid_clock.cc:648] HybridClock initialized: now 1786458895538961 us; error 44 us; skew 500 ppm
Aug 11 14:34:55 dist-test-slave-cbsf krb5kdc[16567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458895, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:55.545228 16581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:55.545792 16581 webserver.cc:460] Webserver started at http://127.0.0.1:37329/ using document root <none> and password file <none>
I20260811 14:34:55.546041 16581 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:55.546113 16581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:55.546270 16581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:55.547420 16581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "af43ede6b02044a788ccedde83488243"
format_stamp: "Formatted at 2026-08-11 14:34:55 on dist-test-slave-cbsf"
server_key: "05f3413d88c54d8e12e0cb467dd3ae95"
server_key_iv: "3ca98f2e47e579193b90c5e2542e6631"
server_key_version: "encryptionkey@0"
I20260811 14:34:55.547811 16581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "af43ede6b02044a788ccedde83488243"
format_stamp: "Formatted at 2026-08-11 14:34:55 on dist-test-slave-cbsf"
server_key: "05f3413d88c54d8e12e0cb467dd3ae95"
server_key_iv: "3ca98f2e47e579193b90c5e2542e6631"
server_key_version: "encryptionkey@0"
I20260811 14:34:55.549539 16581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:55.550359 16596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:55.550567 16581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:55.550684 16581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "af43ede6b02044a788ccedde83488243"
format_stamp: "Formatted at 2026-08-11 14:34:55 on dist-test-slave-cbsf"
server_key: "05f3413d88c54d8e12e0cb467dd3ae95"
server_key_iv: "3ca98f2e47e579193b90c5e2542e6631"
server_key_version: "encryptionkey@0"
I20260811 14:34:55.550776 16581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:55.554418 16581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:55.556875 16581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:55.557041 16581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:55.561486 16581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38667
I20260811 14:34:55.561527 16661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38667 every 8 connection(s)
I20260811 14:34:55.561880 16581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:55.562885 16662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:55.565306 16662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243: Bootstrap starting.
I20260811 14:34:55.565896 16662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:55.566201 16662 log.cc:824] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:55.566937 16662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243: No bootstrap required, opened a new log
I20260811 14:34:55.568540 16662 raft_consensus.cc:348] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af43ede6b02044a788ccedde83488243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260811 14:34:55.568684 16662 raft_consensus.cc:374] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:55.568806 16662 raft_consensus.cc:729] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af43ede6b02044a788ccedde83488243, State: Initialized, Role: FOLLOWER
I20260811 14:34:55.569003 16662 consensus_queue.cc:260] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af43ede6b02044a788ccedde83488243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260811 14:34:55.569092 16662 raft_consensus.cc:388] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:55.569133 16662 raft_consensus.cc:482] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:55.569185 16662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:55.569351 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16581
I20260811 14:34:55.569468 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:55.569662 24299 external_mini_cluster.cc:1428] Setting key 2fd96b17a2ef67a438cae16c57f984bf
I20260811 14:34:55.569952 16662 raft_consensus.cc:504] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af43ede6b02044a788ccedde83488243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260811 14:34:55.570093 16662 leader_election.cc:302] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af43ede6b02044a788ccedde83488243; no voters:
I20260811 14:34:55.570520 16662 leader_election.cc:288] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:55.570556 16668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:55.570662 16668 raft_consensus.cc:686] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [term 1 LEADER]: Becoming Leader. State: Replica: af43ede6b02044a788ccedde83488243, State: Running, Role: LEADER
I20260811 14:34:55.570813 16668 consensus_queue.cc:237] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af43ede6b02044a788ccedde83488243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } }
I20260811 14:34:55.570799 16662 sys_catalog.cc:564] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:55.571412 16669 sys_catalog.cc:455] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af43ede6b02044a788ccedde83488243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af43ede6b02044a788ccedde83488243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } } }
I20260811 14:34:55.571410 16670 sys_catalog.cc:455] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af43ede6b02044a788ccedde83488243. Latest consensus state: current_term: 1 leader_uuid: "af43ede6b02044a788ccedde83488243" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af43ede6b02044a788ccedde83488243" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38667 } } }
I20260811 14:34:55.571544 16669 sys_catalog.cc:458] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:55.571549 16670 sys_catalog.cc:458] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:55.571787 16673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:55.572252 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:55.570181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:50001 (local address 127.0.0.1:38667)
0811 14:34:55.570367 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:55.570375 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:55.570442 (+ 67us) server_negotiation.cc:407] Connection header received
0811 14:34:55.570579 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:55.570587 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:55.570623 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:55.571088 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:55.571558 (+ 470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:55.571562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:55.571562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:55.571567 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:55.571628 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:55.571645 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:55.571748 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:55.572000 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 14:34:55.572050 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260811 14:34:55.572319 16673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:55.573204 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:55.575171 16673 catalog_manager.cc:1347] Generated new cluster ID: 4b526632d5a44726b8bddab4e4e24ab2
I20260811 14:34:55.575229 16673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:55.581924 16667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:55.574823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:43337 (local address 127.0.0.1:38667)
0811 14:34:55.574863 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:55.574865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:55.574927 (+ 62us) server_negotiation.cc:407] Connection header received
0811 14:34:55.580856 (+ 5929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:55.580858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:55.580866 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:55.580952 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:55.581676 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:55.581678 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:55.581678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:55.581683 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:55.581702 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:55.581707 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:55.581774 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:55.581828 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 14:34:55.581844 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:55.596378 16673 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:55.596598 16673 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:55.602176 16673 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af43ede6b02044a788ccedde83488243: Generated new TSK 0
I20260811 14:34:55.602450 16673 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:55.611415 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16581
2026-08-11T14:34:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1424 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:55 dist-test-slave-cbsf krb5kdc[16697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:55 dist-test-slave-cbsf krb5kdc[16697](info): set up 2 sockets
Aug 11 14:34:55 dist-test-slave-cbsf krb5kdc[16697](info): commencing operation
krb5kdc: starting...
W20260811 14:34:56.752255 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:56 dist-test-slave-cbsf krb5kdc[16697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:56.881899 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41961
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:56.966913 16711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:56.967190 16711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:56.967228 16711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:56.968413 16711 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:56.968488 16711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:56.968513 16711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:56.968541 16711 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:56.968572 16711 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:56.970124 16711 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41961
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:56.970431 16711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:56.970669 16711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:56.975734 16718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:56.975740 16720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:56.975744 16717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:56.976068 16711 server_base.cc:1034] running on GCE node
I20260811 14:34:56.976343 16711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:56.976637 16711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:56.977826 16711 hybrid_clock.cc:648] HybridClock initialized: now 1786458896977808 us; error 38 us; skew 500 ppm
Aug 11 14:34:56 dist-test-slave-cbsf krb5kdc[16697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458896, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:56.980300 16711 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:56.980862 16711 webserver.cc:460] Webserver started at http://127.0.0.1:37493/ using document root <none> and password file <none>
I20260811 14:34:56.981117 16711 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:56.981189 16711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:56.981328 16711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:56.982420 16711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "c014adf916e945e1b52df9297d888d56"
format_stamp: "Formatted at 2026-08-11 14:34:56 on dist-test-slave-cbsf"
server_key: "b877086d6d20d99b6792ee478f0ed441"
server_key_iv: "d9830a5b6568574bcda99d02d0367683"
server_key_version: "encryptionkey@0"
I20260811 14:34:56.982795 16711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "c014adf916e945e1b52df9297d888d56"
format_stamp: "Formatted at 2026-08-11 14:34:56 on dist-test-slave-cbsf"
server_key: "b877086d6d20d99b6792ee478f0ed441"
server_key_iv: "d9830a5b6568574bcda99d02d0367683"
server_key_version: "encryptionkey@0"
I20260811 14:34:56.984182 16711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:34:56.984995 16727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:56.985176 16711 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:34:56.985281 16711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "c014adf916e945e1b52df9297d888d56"
format_stamp: "Formatted at 2026-08-11 14:34:56 on dist-test-slave-cbsf"
server_key: "b877086d6d20d99b6792ee478f0ed441"
server_key_iv: "d9830a5b6568574bcda99d02d0367683"
server_key_version: "encryptionkey@0"
I20260811 14:34:56.985364 16711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:57.007018 16711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:57.009197 16711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:57.009366 16711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:57.013734 16711 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41961
I20260811 14:34:57.013792 16786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41961 every 8 connection(s)
I20260811 14:34:57.014194 16711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:57.015117 16788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:57.017660 16788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56: Bootstrap starting.
I20260811 14:34:57.018301 16788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:57.018291 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16711
I20260811 14:34:57.018471 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:57.018652 16788 log.cc:824] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:57.018689 24299 external_mini_cluster.cc:1428] Setting key 925d2247470af3b14db8c46da524fe6b
I20260811 14:34:57.019557 16788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56: No bootstrap required, opened a new log
I20260811 14:34:57.021929 16788 raft_consensus.cc:348] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c014adf916e945e1b52df9297d888d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 14:34:57.022101 16788 raft_consensus.cc:374] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:57.022149 16788 raft_consensus.cc:729] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c014adf916e945e1b52df9297d888d56, State: Initialized, Role: FOLLOWER
I20260811 14:34:57.022285 16788 consensus_queue.cc:260] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c014adf916e945e1b52df9297d888d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 14:34:57.022394 16788 raft_consensus.cc:388] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:57.022439 16788 raft_consensus.cc:482] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:57.022487 16788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:57.023118 16788 raft_consensus.cc:504] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c014adf916e945e1b52df9297d888d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 14:34:57.023249 16788 leader_election.cc:302] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c014adf916e945e1b52df9297d888d56; no voters:
I20260811 14:34:57.023466 16788 leader_election.cc:288] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:57.023496 16793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:57.023702 16793 raft_consensus.cc:686] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [term 1 LEADER]: Becoming Leader. State: Replica: c014adf916e945e1b52df9297d888d56, State: Running, Role: LEADER
I20260811 14:34:57.023821 16793 consensus_queue.cc:237] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c014adf916e945e1b52df9297d888d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } }
I20260811 14:34:57.023883 16788 sys_catalog.cc:564] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:57.024217 16794 sys_catalog.cc:455] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c014adf916e945e1b52df9297d888d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c014adf916e945e1b52df9297d888d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } } }
I20260811 14:34:57.024227 16795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c014adf916e945e1b52df9297d888d56. Latest consensus state: current_term: 1 leader_uuid: "c014adf916e945e1b52df9297d888d56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c014adf916e945e1b52df9297d888d56" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41961 } } }
I20260811 14:34:57.025369 16794 sys_catalog.cc:458] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:57.025625 16795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56 [sys.catalog]: This master's current role is: LEADER
W20260811 14:34:57.025806 16814 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:34:57.025882 16814 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:34:57.025981 16816 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:57.026167 16816 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 14:34:57 dist-test-slave-cbsf krb5kdc[16697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 14:34:57.027840 16816 catalog_manager.cc:1347] Generated new cluster ID: 5b0f7e4de38d4e8bbf74ef40cc2241b3
I20260811 14:34:57.027899 16816 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:57.031092 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:57.019282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:55789 (local address 127.0.0.1:41961)
0811 14:34:57.019497 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:57.019507 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:57.019543 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:34:57.020061 (+ 518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:57.020068 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:57.020122 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:57.020278 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:57.021062 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:57.021956 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:57.022882 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:57.023389 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:57.023508 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:57.027118 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:57.027122 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:57.027123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:57.027136 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:57.029279 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:57.029783 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:57.029787 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:57.029787 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:57.029901 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:57.030151 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:57.030152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:57.030152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:57.030451 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:57.030605 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:57.030767 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 14:34:57.030869 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":100,"threads_started":1}
I20260811 14:34:57.032433 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:57.041945 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:57.034132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:57805 (local address 127.0.0.1:41961)
0811 14:34:57.034174 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:57.034176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:57.034225 (+ 49us) server_negotiation.cc:407] Connection header received
0811 14:34:57.034288 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:57.034289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:57.034299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:57.034370 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:57.035014 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:57.035814 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:57.036828 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:57.037211 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:57.037291 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:57.038721 (+ 1430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:57.038723 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:57.038723 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:34:57.038731 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:57.040535 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:57.041036 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:57.041037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:57.041038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:57.041105 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:34:57.041343 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:34:57.041344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:34:57.041344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:34:57.041490 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:57.041613 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:57.041759 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 14:34:57.041831 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:34:57.051153 16816 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:57.051739 16816 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:57.056349 16816 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c014adf916e945e1b52df9297d888d56: Generated new TSK 0
I20260811 14:34:57.056608 16816 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:57.064620 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16711
W20260811 14:34:57.072099 16695 connection.cc:535] client connection to 127.0.0.1:41961 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41961): Connection reset by peer (error 104)
2026-08-11T14:34:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:57 dist-test-slave-cbsf krb5kdc[16826](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:57 dist-test-slave-cbsf krb5kdc[16826](info): set up 2 sockets
Aug 11 14:34:57 dist-test-slave-cbsf krb5kdc[16826](info): commencing operation
krb5kdc: starting...
W20260811 14:34:58.211019 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:58 dist-test-slave-cbsf krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:58.346263 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:58.432029 16842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:58.432250 16842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:58.432276 16842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:58.433395 16842 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:58.433442 16842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:58.433456 16842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:58.433470 16842 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:58.433481 16842 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:58.434968 16842 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:58.435194 16842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:58.435369 16842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:58.440387 16849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:58.440409 16853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:58.440460 16848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:58.440639 16842 server_base.cc:1034] running on GCE node
I20260811 14:34:58.440907 16842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:58.441182 16842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:58.442322 16842 hybrid_clock.cc:648] HybridClock initialized: now 1786458898442290 us; error 52 us; skew 500 ppm
Aug 11 14:34:58 dist-test-slave-cbsf krb5kdc[16826](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458898, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:58.445087 16842 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:58.445573 16842 webserver.cc:460] Webserver started at http://127.0.0.1:41531/ using document root <none> and password file <none>
I20260811 14:34:58.445834 16842 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:58.445881 16842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:58.446058 16842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:58.447212 16842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "a18881681a684e9ca01c0b9c1f4b1284"
format_stamp: "Formatted at 2026-08-11 14:34:58 on dist-test-slave-cbsf"
server_key: "998395ffb82bb8e7ca423b46edb594a0"
server_key_iv: "3059aea0943742a0fd9fa685edde1b88"
server_key_version: "encryptionkey@0"
I20260811 14:34:58.447592 16842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "a18881681a684e9ca01c0b9c1f4b1284"
format_stamp: "Formatted at 2026-08-11 14:34:58 on dist-test-slave-cbsf"
server_key: "998395ffb82bb8e7ca423b46edb594a0"
server_key_iv: "3059aea0943742a0fd9fa685edde1b88"
server_key_version: "encryptionkey@0"
I20260811 14:34:58.449136 16842 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:34:58.449993 16860 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:58.450196 16842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:58.450322 16842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "a18881681a684e9ca01c0b9c1f4b1284"
format_stamp: "Formatted at 2026-08-11 14:34:58 on dist-test-slave-cbsf"
server_key: "998395ffb82bb8e7ca423b46edb594a0"
server_key_iv: "3059aea0943742a0fd9fa685edde1b88"
server_key_version: "encryptionkey@0"
I20260811 14:34:58.450405 16842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:58.471884 16842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:58.474584 16842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:58.474794 16842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:58.479182 16842 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33279
I20260811 14:34:58.479223 16917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33279 every 8 connection(s)
I20260811 14:34:58.479796 16842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:58.480562 16918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:58.482621 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16842
I20260811 14:34:58.482770 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:58.482980 24299 external_mini_cluster.cc:1428] Setting key b3a9bfd5920192cde068116cc79fbe8a
I20260811 14:34:58.483305 16918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284: Bootstrap starting.
I20260811 14:34:58.484092 16918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:58.484421 16918 log.cc:824] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:58.485280 16918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284: No bootstrap required, opened a new log
I20260811 14:34:58.487696 16918 raft_consensus.cc:348] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18881681a684e9ca01c0b9c1f4b1284" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260811 14:34:58.487906 16918 raft_consensus.cc:374] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:58.487964 16918 raft_consensus.cc:729] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a18881681a684e9ca01c0b9c1f4b1284, State: Initialized, Role: FOLLOWER
I20260811 14:34:58.488149 16918 consensus_queue.cc:260] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18881681a684e9ca01c0b9c1f4b1284" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260811 14:34:58.488270 16918 raft_consensus.cc:388] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:58.488305 16918 raft_consensus.cc:482] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:58.488359 16918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:58.489189 16918 raft_consensus.cc:504] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18881681a684e9ca01c0b9c1f4b1284" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260811 14:34:58.489287 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:58.483645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:53317 (local address 127.0.0.1:33279)
0811 14:34:58.483841 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:58.483850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:58.483879 (+ 29us) server_negotiation.cc:407] Connection header received
0811 14:34:58.484409 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:58.484418 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:58.484471 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:58.484642 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:58.484916 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:58.485925 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:58.487159 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:58.487592 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:58.487693 (+ 101us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:58.488452 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:58.488457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:58.488458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:58.488665 (+ 207us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:58.488754 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:58.488771 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:58.488854 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:58.489022 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 14:34:58.489057 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260811 14:34:58.489290 16918 leader_election.cc:302] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a18881681a684e9ca01c0b9c1f4b1284; no voters:
I20260811 14:34:58.489568 16918 leader_election.cc:288] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:58.489602 16923 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:58.489782 16923 raft_consensus.cc:686] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [term 1 LEADER]: Becoming Leader. State: Replica: a18881681a684e9ca01c0b9c1f4b1284, State: Running, Role: LEADER
I20260811 14:34:58.489871 16923 consensus_queue.cc:237] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18881681a684e9ca01c0b9c1f4b1284" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } }
I20260811 14:34:58.489917 16918 sys_catalog.cc:564] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:58.490353 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:58.490235 16924 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a18881681a684e9ca01c0b9c1f4b1284. Latest consensus state: current_term: 1 leader_uuid: "a18881681a684e9ca01c0b9c1f4b1284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18881681a684e9ca01c0b9c1f4b1284" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } } }
I20260811 14:34:58.490432 16924 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:58.490347 16925 sys_catalog.cc:455] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a18881681a684e9ca01c0b9c1f4b1284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a18881681a684e9ca01c0b9c1f4b1284" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33279 } } }
I20260811 14:34:58.490639 16925 sys_catalog.cc:458] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:58.491007 16931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:58.491549 16931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:58.493382 16931 catalog_manager.cc:1347] Generated new cluster ID: 3d372d6312ff4c81bebe0ce94396af9a
I20260811 14:34:58.493441 16931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:58.496043 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:58.492192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:58793 (local address 127.0.0.1:33279)
0811 14:34:58.492237 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:58.492240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:58.492442 (+ 202us) server_negotiation.cc:407] Connection header received
0811 14:34:58.492544 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:58.492545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:58.492556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:58.492587 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:58.492817 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:58.493532 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:58.494589 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:34:58.494909 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:34:58.494984 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:34:58.495766 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:58.495767 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:58.495767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:58.495773 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:58.495790 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:58.495794 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:58.495851 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:58.495957 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 14:34:58.495968 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:34:58.510916 16931 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:58.511487 16931 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:58.520121 16931 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a18881681a684e9ca01c0b9c1f4b1284: Generated new TSK 0
I20260811 14:34:58.520335 16931 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:58.524343 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16842
W20260811 14:34:58.531232 16824 connection.cc:535] client connection to 127.0.0.1:33279 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33279): Connection reset by peer (error 104)
2026-08-11T14:34:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1458 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:58 dist-test-slave-cbsf krb5kdc[16954](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:58 dist-test-slave-cbsf krb5kdc[16954](info): set up 2 sockets
Aug 11 14:34:58 dist-test-slave-cbsf krb5kdc[16954](info): commencing operation
krb5kdc: starting...
W20260811 14:34:59.647534 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:34:59 dist-test-slave-cbsf krb5kdc[16954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:34:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:34:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 14:34:59.776039 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:34:59.861085 16968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:34:59.861325 16968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:34:59.861351 16968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:34:59.862584 16968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:34:59.862643 16968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:34:59.862658 16968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:34:59.862671 16968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:34:59.862684 16968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:34:59.864251 16968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:34:59.864537 16968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:34:59.864838 16968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:34:59.870010 16977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:59.870052 16968 server_base.cc:1034] running on GCE node
W20260811 14:34:59.870010 16975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:34:59.870033 16974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:34:59.870530 16968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:34:59.870858 16968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:34:59.872031 16968 hybrid_clock.cc:648] HybridClock initialized: now 1786458899872020 us; error 44 us; skew 500 ppm
Aug 11 14:34:59 dist-test-slave-cbsf krb5kdc[16954](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458899, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:34:59.874715 16968 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 14:34:59.875192 16968 webserver.cc:460] Webserver started at http://127.0.0.1:36973/ using document root <none> and password file <none>
I20260811 14:34:59.875429 16968 fs_manager.cc:359] Metadata directory not provided
I20260811 14:34:59.875478 16968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:34:59.875643 16968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:34:59.876806 16968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "57ac80896bb94d339f5e63f0d92d29f1"
format_stamp: "Formatted at 2026-08-11 14:34:59 on dist-test-slave-cbsf"
server_key: "467a0ce614a8ab08ee2cc337bf043fd9"
server_key_iv: "540654c57bed87df0355a3b4aaded768"
server_key_version: "encryptionkey@0"
I20260811 14:34:59.877198 16968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "57ac80896bb94d339f5e63f0d92d29f1"
format_stamp: "Formatted at 2026-08-11 14:34:59 on dist-test-slave-cbsf"
server_key: "467a0ce614a8ab08ee2cc337bf043fd9"
server_key_iv: "540654c57bed87df0355a3b4aaded768"
server_key_version: "encryptionkey@0"
I20260811 14:34:59.878603 16968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260811 14:34:59.879439 16986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:34:59.879648 16968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:34:59.879762 16968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "57ac80896bb94d339f5e63f0d92d29f1"
format_stamp: "Formatted at 2026-08-11 14:34:59 on dist-test-slave-cbsf"
server_key: "467a0ce614a8ab08ee2cc337bf043fd9"
server_key_iv: "540654c57bed87df0355a3b4aaded768"
server_key_version: "encryptionkey@0"
I20260811 14:34:59.879840 16968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:34:59.883371 16968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:34:59.885795 16968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:34:59.885963 16968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:34:59.890241 16968 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45299
I20260811 14:34:59.890275 17044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45299 every 8 connection(s)
I20260811 14:34:59.890642 16968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:34:59.891652 17045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:34:59.891934 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 16968
I20260811 14:34:59.892060 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:34:59.892279 24299 external_mini_cluster.cc:1428] Setting key 6c5026cc3e828122c406e91d952e15f3
I20260811 14:34:59.894662 17045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1: Bootstrap starting.
I20260811 14:34:59.894903 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:59.892827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:49443 (local address 127.0.0.1:45299)
0811 14:34:59.893048 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:59.893065 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:59.893091 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:34:59.893520 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:59.893526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:59.893550 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:59.893705 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:59.894395 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:59.894398 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:59.894398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:59.894402 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:59.894450 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:59.894461 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:59.894538 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:59.894606 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 14:34:59.894650 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":84,"threads_started":1}
I20260811 14:34:59.895424 17045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1: Neither blocks nor log segments found. Creating new log.
I20260811 14:34:59.895780 17045 log.cc:824] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:34:59.895955 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:34:59.896530 17045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1: No bootstrap required, opened a new log
I20260811 14:34:59.898685 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:34:59.897504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.129:49753 (local address 127.0.0.1:45299)
0811 14:34:59.897544 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:34:59.897546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:34:59.897625 (+ 79us) server_negotiation.cc:407] Connection header received
0811 14:34:59.897711 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:34:59.897712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:34:59.897718 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:34:59.897776 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:34:59.898454 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:34:59.898456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:34:59.898456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:34:59.898461 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:34:59.898476 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:34:59.898481 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:34:59.898539 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 14:34:59.898601 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 14:34:59.898616 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:34:59.899071 17045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac80896bb94d339f5e63f0d92d29f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260811 14:34:59.899271 17045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:34:59.899312 17045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ac80896bb94d339f5e63f0d92d29f1, State: Initialized, Role: FOLLOWER
I20260811 14:34:59.899436 17045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac80896bb94d339f5e63f0d92d29f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260811 14:34:59.899550 17045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:34:59.899587 17045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:34:59.899628 17045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:34:59.900528 17045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac80896bb94d339f5e63f0d92d29f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260811 14:34:59.900657 17045 leader_election.cc:302] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57ac80896bb94d339f5e63f0d92d29f1; no voters:
I20260811 14:34:59.900873 17045 leader_election.cc:288] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:34:59.900954 17055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:34:59.901115 17055 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [term 1 LEADER]: Becoming Leader. State: Replica: 57ac80896bb94d339f5e63f0d92d29f1, State: Running, Role: LEADER
I20260811 14:34:59.901146 17045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:34:59.901223 17055 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac80896bb94d339f5e63f0d92d29f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } }
I20260811 14:34:59.901923 17056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57ac80896bb94d339f5e63f0d92d29f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac80896bb94d339f5e63f0d92d29f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } } }
I20260811 14:34:59.902169 17056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:59.901865 17057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ac80896bb94d339f5e63f0d92d29f1. Latest consensus state: current_term: 1 leader_uuid: "57ac80896bb94d339f5e63f0d92d29f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac80896bb94d339f5e63f0d92d29f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45299 } } }
I20260811 14:34:59.902300 17057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:34:59.902551 17065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:34:59.903123 17065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:34:59.904728 17065 catalog_manager.cc:1347] Generated new cluster ID: 23ed27845710415f9c6084702def3647
I20260811 14:34:59.904786 17065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:34:59.921808 17065 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:34:59.922039 17065 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:34:59.927004 17065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57ac80896bb94d339f5e63f0d92d29f1: Generated new TSK 0
I20260811 14:34:59.927218 17065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:34:59.933054 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 16968
2026-08-11T14:34:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:34:59 dist-test-slave-cbsf krb5kdc[17077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:34:59 dist-test-slave-cbsf krb5kdc[17077](info): set up 2 sockets
Aug 11 14:34:59 dist-test-slave-cbsf krb5kdc[17077](info): commencing operation
krb5kdc: starting...
W20260811 14:35:01.062652 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:01 dist-test-slave-cbsf krb5kdc[17077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:01.193387 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:43033
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:40895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:43033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:01.278524 17093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:01.278782 17093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:01.278820 17093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:01.279978 17093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:01.280048 17093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:01.280072 17093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:01.280102 17093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:01.280134 17093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:01.281670 17093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:40895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:43033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:43033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:01.281965 17093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:01.282215 17093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:01.287237 17101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:01.287247 17099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:01.287247 17103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:01.287335 17093 server_base.cc:1034] running on GCE node
I20260811 14:35:01.287783 17093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:01.288105 17093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:01.289309 17093 hybrid_clock.cc:648] HybridClock initialized: now 1786458901289287 us; error 40 us; skew 500 ppm
Aug 11 14:35:01 dist-test-slave-cbsf krb5kdc[17077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458901, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:01.297240 17093 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:01.297755 17093 webserver.cc:460] Webserver started at http://127.23.186.254:34067/ using document root <none> and password file <none>
I20260811 14:35:01.298030 17093 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:01.298106 17093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:01.298261 17093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:01.299444 17093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "d4967ac42030485493b6f3a1a41e1374"
format_stamp: "Formatted at 2026-08-11 14:35:01 on dist-test-slave-cbsf"
server_key: "d907fb7e672dbcdcc4d6097da6b0ccd4"
server_key_iv: "6e9ecf7f3a042eee61590480466301b7"
server_key_version: "encryptionkey@0"
I20260811 14:35:01.299847 17093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "d4967ac42030485493b6f3a1a41e1374"
format_stamp: "Formatted at 2026-08-11 14:35:01 on dist-test-slave-cbsf"
server_key: "d907fb7e672dbcdcc4d6097da6b0ccd4"
server_key_iv: "6e9ecf7f3a042eee61590480466301b7"
server_key_version: "encryptionkey@0"
I20260811 14:35:01.301374 17093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:01.302173 17112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:01.302374 17093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:01.302488 17093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "d4967ac42030485493b6f3a1a41e1374"
format_stamp: "Formatted at 2026-08-11 14:35:01 on dist-test-slave-cbsf"
server_key: "d907fb7e672dbcdcc4d6097da6b0ccd4"
server_key_iv: "6e9ecf7f3a042eee61590480466301b7"
server_key_version: "encryptionkey@0"
I20260811 14:35:01.302582 17093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:01.323472 17093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:01.325708 17093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:01.325863 17093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:01.330893 17093 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:43033
I20260811 14:35:01.330993 17176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:43033 every 8 connection(s)
I20260811 14:35:01.331362 17093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:01.332263 17178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:01.334659 17178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374: Bootstrap starting.
I20260811 14:35:01.335271 17178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:01.335575 17178 log.cc:824] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:01.336320 17178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374: No bootstrap required, opened a new log
I20260811 14:35:01.338032 17178 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4967ac42030485493b6f3a1a41e1374" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43033 } }
I20260811 14:35:01.338174 17178 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:01.338197 17178 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4967ac42030485493b6f3a1a41e1374, State: Initialized, Role: FOLLOWER
I20260811 14:35:01.338382 17178 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4967ac42030485493b6f3a1a41e1374" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43033 } }
I20260811 14:35:01.338445 17178 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:01.338507 17178 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:01.338560 17178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:01.339449 17178 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4967ac42030485493b6f3a1a41e1374" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43033 } }
I20260811 14:35:01.339586 17178 leader_election.cc:302] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4967ac42030485493b6f3a1a41e1374; no voters:
I20260811 14:35:01.339799 17178 leader_election.cc:288] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:01.339823 17181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:01.339937 17181 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [term 1 LEADER]: Becoming Leader. State: Replica: d4967ac42030485493b6f3a1a41e1374, State: Running, Role: LEADER
I20260811 14:35:01.339924 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17093
I20260811 14:35:01.340056 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:01.340075 17178 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:01.340104 17181 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4967ac42030485493b6f3a1a41e1374" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43033 } }
I20260811 14:35:01.340283 24299 external_mini_cluster.cc:1428] Setting key f32dd1544d0796f6eefc23578c9ae6fe
I20260811 14:35:01.340469 17183 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4967ac42030485493b6f3a1a41e1374" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4967ac42030485493b6f3a1a41e1374" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43033 } } }
I20260811 14:35:01.340554 17183 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:01.340775 17182 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4967ac42030485493b6f3a1a41e1374. Latest consensus state: current_term: 1 leader_uuid: "d4967ac42030485493b6f3a1a41e1374" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4967ac42030485493b6f3a1a41e1374" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 43033 } } }
I20260811 14:35:01.340843 17182 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:01.341393 17187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:01.341986 17187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:01.343765 17187 catalog_manager.cc:1347] Generated new cluster ID: 847f46e29e814e1abe1891db0a09d737
I20260811 14:35:01.343817 17187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:01 dist-test-slave-cbsf krb5kdc[17077](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:01.352619 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:01.340785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52181 (local address 127.23.186.254:43033)
0811 14:35:01.341052 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:01.341064 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:01.341243 (+ 179us) server_negotiation.cc:407] Connection header received
0811 14:35:01.341408 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:01.341416 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:01.341503 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:01.341643 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:01.342612 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:01.344023 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:01.345137 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:01.345775 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:01.345896 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:01.348817 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:01.348821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:01.348822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:01.348835 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:01.350826 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:01.351305 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:01.351307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:01.351308 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:01.351425 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:01.351671 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:01.351672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:01.351672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:01.351992 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:01.352149 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:01.352275 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 14:35:01.352371 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":91,"threads_started":1}
I20260811 14:35:01.353652 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:01.358019 17187 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:01.358733 17187 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:01.362984 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:01.355324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34705 (local address 127.23.186.254:43033)
0811 14:35:01.355357 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:01.355359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:01.355368 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:01.355416 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:01.355416 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:01.355425 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:01.355478 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:01.356103 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:01.357127 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:01.358175 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:01.358472 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:01.358543 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:01.360023 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:01.360025 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:01.360025 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:01.360032 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:01.361619 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:01.362050 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:01.362051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:01.362051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:01.362107 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:01.362339 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:01.362340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:01.362341 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:01.362459 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:01.362552 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:01.362716 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 14:35:01.362781 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":270,"server-negotiator.queue_time_us":16}
I20260811 14:35:01.366334 17187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4967ac42030485493b6f3a1a41e1374: Generated new TSK 0
I20260811 14:35:01.366557 17187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:01.373824 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17093
W20260811 14:35:01.384146 17075 connection.cc:535] client connection to 127.23.186.254:43033 recv error: Network error: failed to read from TLS socket (remote: 127.23.186.254:43033): Connection reset by peer (error 104)
2026-08-11T14:35:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:01 dist-test-slave-cbsf krb5kdc[17212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:01 dist-test-slave-cbsf krb5kdc[17212](info): set up 2 sockets
Aug 11 14:35:01 dist-test-slave-cbsf krb5kdc[17212](info): commencing operation
krb5kdc: starting...
W20260811 14:35:02.499300 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:02 dist-test-slave-cbsf krb5kdc[17212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:02.631793 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39737
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:37125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:02.716657 17225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:02.716917 17225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:02.716941 17225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:02.718122 17225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:02.718168 17225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:02.718183 17225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:02.718195 17225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:02.718209 17225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:02.719779 17225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:37125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:02.719997 17225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:02.720185 17225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:02.725328 17230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:02.725358 17233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:02.725457 17225 server_base.cc:1034] running on GCE node
W20260811 14:35:02.725328 17231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:02.725880 17225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:02.726190 17225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:02.727347 17225 hybrid_clock.cc:648] HybridClock initialized: now 1786458902727342 us; error 39 us; skew 500 ppm
Aug 11 14:35:02 dist-test-slave-cbsf krb5kdc[17212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458902, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:02.729907 17225 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:02.730429 17225 webserver.cc:460] Webserver started at http://127.23.186.254:34423/ using document root <none> and password file <none>
I20260811 14:35:02.730679 17225 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:02.730751 17225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:02.730909 17225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:02.732012 17225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "ed6fa82862204de0803c0706d0413cdd"
format_stamp: "Formatted at 2026-08-11 14:35:02 on dist-test-slave-cbsf"
server_key: "7cab5b42fe102a4314674b36b45e848a"
server_key_iv: "ea73e3c53bfea80a5306430f9d05539b"
server_key_version: "encryptionkey@0"
I20260811 14:35:02.732390 17225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "ed6fa82862204de0803c0706d0413cdd"
format_stamp: "Formatted at 2026-08-11 14:35:02 on dist-test-slave-cbsf"
server_key: "7cab5b42fe102a4314674b36b45e848a"
server_key_iv: "ea73e3c53bfea80a5306430f9d05539b"
server_key_version: "encryptionkey@0"
I20260811 14:35:02.733858 17225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260811 14:35:02.734669 17241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:02.734882 17225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:02.734997 17225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "ed6fa82862204de0803c0706d0413cdd"
format_stamp: "Formatted at 2026-08-11 14:35:02 on dist-test-slave-cbsf"
server_key: "7cab5b42fe102a4314674b36b45e848a"
server_key_iv: "ea73e3c53bfea80a5306430f9d05539b"
server_key_version: "encryptionkey@0"
I20260811 14:35:02.735077 17225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:02.756106 17225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:02.758385 17225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:02.758572 17225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:02.762825 17225 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:39737
I20260811 14:35:02.762867 17308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:39737 every 8 connection(s)
I20260811 14:35:02.763257 17225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:02.764137 17309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:02.766520 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd: Bootstrap starting.
I20260811 14:35:02.767069 17309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:02.767313 17309 log.cc:824] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:02.768010 17309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd: No bootstrap required, opened a new log
I20260811 14:35:02.768033 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17225
I20260811 14:35:02.768200 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:02.768390 24299 external_mini_cluster.cc:1428] Setting key 56817168d43a00693e4d611c9e74aea0
I20260811 14:35:02.770066 17309 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6fa82862204de0803c0706d0413cdd" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39737 } }
I20260811 14:35:02.770241 17309 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:02.770280 17309 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed6fa82862204de0803c0706d0413cdd, State: Initialized, Role: FOLLOWER
I20260811 14:35:02.770484 17309 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6fa82862204de0803c0706d0413cdd" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39737 } }
I20260811 14:35:02.770601 17309 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:02.770643 17309 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:02.770695 17309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:02.771281 17309 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6fa82862204de0803c0706d0413cdd" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39737 } }
I20260811 14:35:02.771405 17309 leader_election.cc:302] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed6fa82862204de0803c0706d0413cdd; no voters:
I20260811 14:35:02.771607 17309 leader_election.cc:288] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:02.771657 17314 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:02.771793 17314 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [term 1 LEADER]: Becoming Leader. State: Replica: ed6fa82862204de0803c0706d0413cdd, State: Running, Role: LEADER
I20260811 14:35:02.771885 17309 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:02.771930 17314 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6fa82862204de0803c0706d0413cdd" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39737 } }
I20260811 14:35:02.773051 17316 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed6fa82862204de0803c0706d0413cdd. Latest consensus state: current_term: 1 leader_uuid: "ed6fa82862204de0803c0706d0413cdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6fa82862204de0803c0706d0413cdd" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39737 } } }
I20260811 14:35:02.773140 17316 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:02.773346 17332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:02.773684 17315 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed6fa82862204de0803c0706d0413cdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed6fa82862204de0803c0706d0413cdd" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39737 } } }
I20260811 14:35:02.773782 17315 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:02.774085 17332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:02.775714 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:02.768987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53153 (local address 127.23.186.254:39737)
0811 14:35:02.769180 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:02.769188 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:02.769214 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:02.770039 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:02.770046 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:02.770104 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:02.770219 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:02.770517 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:02.772016 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:02.773328 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:02.774023 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:02.774134 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:02.774957 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:02.774962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:02.774962 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:02.775186 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:02.775224 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:02.775240 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:02.775335 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:02.775453 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 14:35:02.775485 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
I20260811 14:35:02.776225 17332 catalog_manager.cc:1347] Generated new cluster ID: 15ad065f18fc436e942f914b37102dd5
I20260811 14:35:02.776326 17332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:02.776708 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:02.785386 17332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:02.786080 17332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:02.788903 17313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:02.785007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60459 (local address 127.23.186.254:39737)
0811 14:35:02.785056 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:02.785058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:02.785230 (+ 172us) server_negotiation.cc:407] Connection header received
0811 14:35:02.785380 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:02.785381 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:02.785404 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:02.785466 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:02.785683 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:02.786419 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:02.787433 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:02.787779 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:02.787851 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:02.788597 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:02.788598 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:02.788599 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:02.788604 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:02.788622 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:02.788627 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:02.788718 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:02.788808 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 14:35:02.788823 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:02.793726 17332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed6fa82862204de0803c0706d0413cdd: Generated new TSK 0
I20260811 14:35:02.793917 17332 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:02.798296 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17225
W20260811 14:35:02.807091 17210 connection.cc:535] client connection to 127.23.186.254:39737 recv error: Network error: failed to read from TLS socket (remote: 127.23.186.254:39737): Connection reset by peer (error 104)
2026-08-11T14:35:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:02 dist-test-slave-cbsf krb5kdc[17343](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:02 dist-test-slave-cbsf krb5kdc[17343](info): set up 2 sockets
Aug 11 14:35:02 dist-test-slave-cbsf krb5kdc[17343](info): commencing operation
krb5kdc: starting...
W20260811 14:35:03.930935 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:04 dist-test-slave-cbsf krb5kdc[17343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:04.063488 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39717
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:39717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:04.149806 17358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:04.150039 17358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:04.150064 17358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:04.151324 17358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:04.151377 17358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:04.151392 17358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:04.151405 17358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:04.151465 17358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:04.153077 17358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:39717
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:39717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:04.153365 17358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:04.153625 17358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:04.158805 17364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:04.158805 17367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:04.158808 17365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:04.158841 17358 server_base.cc:1034] running on GCE node
I20260811 14:35:04.159248 17358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:04.159560 17358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:04.160781 17358 hybrid_clock.cc:648] HybridClock initialized: now 1786458904160736 us; error 59 us; skew 500 ppm
Aug 11 14:35:04 dist-test-slave-cbsf krb5kdc[17343](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458904, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:04.163642 17358 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:04.164172 17358 webserver.cc:460] Webserver started at http://127.23.186.254:42777/ using document root <none> and password file <none>
I20260811 14:35:04.164379 17358 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:04.164428 17358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:04.164530 17358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:04.165729 17358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "9343712827b54e5ca6cf00fc38002560"
format_stamp: "Formatted at 2026-08-11 14:35:04 on dist-test-slave-cbsf"
server_key: "43a8769c256eb499124d7a6bd9d16108"
server_key_iv: "5cdb90f79cf87f0e38465f5de34a227a"
server_key_version: "encryptionkey@0"
I20260811 14:35:04.166105 17358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "9343712827b54e5ca6cf00fc38002560"
format_stamp: "Formatted at 2026-08-11 14:35:04 on dist-test-slave-cbsf"
server_key: "43a8769c256eb499124d7a6bd9d16108"
server_key_iv: "5cdb90f79cf87f0e38465f5de34a227a"
server_key_version: "encryptionkey@0"
I20260811 14:35:04.167590 17358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:04.168455 17376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:04.168676 17358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:04.168855 17358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "9343712827b54e5ca6cf00fc38002560"
format_stamp: "Formatted at 2026-08-11 14:35:04 on dist-test-slave-cbsf"
server_key: "43a8769c256eb499124d7a6bd9d16108"
server_key_iv: "5cdb90f79cf87f0e38465f5de34a227a"
server_key_version: "encryptionkey@0"
I20260811 14:35:04.168955 17358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:04.172678 17358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:04.175444 17358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:04.175640 17358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:04.180410 17358 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:39717
I20260811 14:35:04.180442 17436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:39717 every 8 connection(s)
I20260811 14:35:04.180941 17358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:04.182003 17437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:04.184448 17437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560: Bootstrap starting.
I20260811 14:35:04.185134 17437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:04.185438 17437 log.cc:824] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:04.186226 17437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560: No bootstrap required, opened a new log
I20260811 14:35:04.187860 17437 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9343712827b54e5ca6cf00fc38002560" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39717 } }
I20260811 14:35:04.188007 17437 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:04.188038 17437 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9343712827b54e5ca6cf00fc38002560, State: Initialized, Role: FOLLOWER
I20260811 14:35:04.188226 17437 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9343712827b54e5ca6cf00fc38002560" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39717 } }
I20260811 14:35:04.188295 17437 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:04.188318 17437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:04.188383 17437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:04.189446 17437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9343712827b54e5ca6cf00fc38002560" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39717 } }
I20260811 14:35:04.189579 17437 leader_election.cc:302] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9343712827b54e5ca6cf00fc38002560; no voters:
I20260811 14:35:04.189633 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17358
I20260811 14:35:04.189767 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:04.189769 17437 leader_election.cc:288] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:04.189850 17440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:04.189977 24299 external_mini_cluster.cc:1428] Setting key 69825cb60f449eb338675041f3fb4b22
I20260811 14:35:04.190057 17437 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:04.190109 17440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [term 1 LEADER]: Becoming Leader. State: Replica: 9343712827b54e5ca6cf00fc38002560, State: Running, Role: LEADER
I20260811 14:35:04.190213 17440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9343712827b54e5ca6cf00fc38002560" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39717 } }
I20260811 14:35:04.190828 17441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9343712827b54e5ca6cf00fc38002560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9343712827b54e5ca6cf00fc38002560" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39717 } } }
I20260811 14:35:04.190944 17441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:04.191179 17442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9343712827b54e5ca6cf00fc38002560. Latest consensus state: current_term: 1 leader_uuid: "9343712827b54e5ca6cf00fc38002560" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9343712827b54e5ca6cf00fc38002560" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 39717 } } }
I20260811 14:35:04.191176 17451 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:04.191905 17451 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:04.192576 17442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:04.193677 17451 catalog_manager.cc:1347] Generated new cluster ID: 65325012521d4563856f225203dfc72d
I20260811 14:35:04.193699 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:04.190770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37301 (local address 127.23.186.254:39717)
0811 14:35:04.190966 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:04.190975 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:04.191010 (+ 35us) server_negotiation.cc:407] Connection header received
0811 14:35:04.191694 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:04.191704 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:04.191737 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:04.191857 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:04.193101 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:04.193105 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:04.193105 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:04.193112 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:04.193162 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:04.193179 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:04.193253 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:04.193450 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 14:35:04.193499 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":81,"threads_started":1}
I20260811 14:35:04.193725 17451 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:04.197749 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:04.206104 17447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:04.201120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52173 (local address 127.23.186.254:39717)
0811 14:35:04.201167 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:04.201169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:04.204935 (+ 3766us) server_negotiation.cc:407] Connection header received
0811 14:35:04.205027 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:04.205028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:04.205035 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:04.205111 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:04.205809 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:04.205811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:04.205811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:04.205816 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:04.205834 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:04.205839 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:04.205912 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:04.205975 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 14:35:04.205991 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:35:04.209882 17451 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:04.210094 17451 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:04.212209 17451 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9343712827b54e5ca6cf00fc38002560: Generated new TSK 0
I20260811 14:35:04.212396 17451 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:04.218495 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17358
2026-08-11T14:35:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:04 dist-test-slave-cbsf krb5kdc[17472](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:04 dist-test-slave-cbsf krb5kdc[17472](info): set up 2 sockets
Aug 11 14:35:04 dist-test-slave-cbsf krb5kdc[17472](info): commencing operation
krb5kdc: starting...
W20260811 14:35:05.344107 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:05 dist-test-slave-cbsf krb5kdc[17472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:05.480028 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41313
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:38059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:41313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:05.566910 17487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:05.567143 17487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:05.567169 17487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:05.568343 17487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:05.568401 17487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:05.568418 17487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:05.568432 17487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:05.568492 17487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:05.570109 17487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:38059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:41313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:41313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:05.570399 17487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:05.570652 17487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:05.575798 17495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:05.575819 17493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:05.575826 17492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:05.575958 17487 server_base.cc:1034] running on GCE node
I20260811 14:35:05.576298 17487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:05.576597 17487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:05.577848 17487 hybrid_clock.cc:648] HybridClock initialized: now 1786458905577844 us; error 65 us; skew 500 ppm
Aug 11 14:35:05 dist-test-slave-cbsf krb5kdc[17472](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458905, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:05.580444 17487 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:05.580991 17487 webserver.cc:460] Webserver started at http://127.23.186.254:44633/ using document root <none> and password file <none>
I20260811 14:35:05.581248 17487 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:05.581321 17487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:05.581494 17487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:05.582599 17487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "2cf4bc39bc574178b84ffc195c9b8eef"
format_stamp: "Formatted at 2026-08-11 14:35:05 on dist-test-slave-cbsf"
server_key: "e7f15203caea209c8d03ea4a2dba32e0"
server_key_iv: "4e5af9dd8f58cca2947bbc65409133b3"
server_key_version: "encryptionkey@0"
I20260811 14:35:05.582983 17487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "2cf4bc39bc574178b84ffc195c9b8eef"
format_stamp: "Formatted at 2026-08-11 14:35:05 on dist-test-slave-cbsf"
server_key: "e7f15203caea209c8d03ea4a2dba32e0"
server_key_iv: "4e5af9dd8f58cca2947bbc65409133b3"
server_key_version: "encryptionkey@0"
I20260811 14:35:05.584374 17487 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:05.585213 17503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:05.585391 17487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:05.585516 17487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "2cf4bc39bc574178b84ffc195c9b8eef"
format_stamp: "Formatted at 2026-08-11 14:35:05 on dist-test-slave-cbsf"
server_key: "e7f15203caea209c8d03ea4a2dba32e0"
server_key_iv: "4e5af9dd8f58cca2947bbc65409133b3"
server_key_version: "encryptionkey@0"
I20260811 14:35:05.585595 17487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:05.600669 17487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:05.602929 17487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:05.603108 17487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:05.607501 17487 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:41313
I20260811 14:35:05.607550 17561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:41313 every 8 connection(s)
I20260811 14:35:05.607949 17487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:05.608865 17562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:05.611199 17562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef: Bootstrap starting.
I20260811 14:35:05.611812 17562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:05.612118 17562 log.cc:824] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:05.612906 17562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef: No bootstrap required, opened a new log
I20260811 14:35:05.614487 17562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41313 } }
I20260811 14:35:05.614630 17562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:05.614653 17562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cf4bc39bc574178b84ffc195c9b8eef, State: Initialized, Role: FOLLOWER
I20260811 14:35:05.614816 17562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41313 } }
I20260811 14:35:05.614907 17562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:05.614946 17562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:05.615027 17562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:05.615921 17562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41313 } }
I20260811 14:35:05.616056 17562 leader_election.cc:302] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2cf4bc39bc574178b84ffc195c9b8eef; no voters:
I20260811 14:35:05.616245 17562 leader_election.cc:288] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:05.616309 17565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:05.616516 17562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:05.616521 17565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [term 1 LEADER]: Becoming Leader. State: Replica: 2cf4bc39bc574178b84ffc195c9b8eef, State: Running, Role: LEADER
I20260811 14:35:05.616696 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17487
I20260811 14:35:05.616832 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:05.616839 17565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41313 } }
I20260811 14:35:05.617048 24299 external_mini_cluster.cc:1428] Setting key cddb7829e0c00ab6a729c060079018ca
I20260811 14:35:05.617274 17566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41313 } } }
I20260811 14:35:05.617362 17566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:05.617727 17575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:05.617759 17567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2cf4bc39bc574178b84ffc195c9b8eef. Latest consensus state: current_term: 1 leader_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2cf4bc39bc574178b84ffc195c9b8eef" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 41313 } } }
I20260811 14:35:05.617883 17567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:05.618531 17575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:05.621208 17575 catalog_manager.cc:1347] Generated new cluster ID: 40a9fbeba9c849f2af91e31afaf8614b
I20260811 14:35:05.621272 17575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:05 dist-test-slave-cbsf krb5kdc[17472](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:05.629297 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:05.617792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56507 (local address 127.23.186.254:41313)
0811 14:35:05.618201 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:05.618211 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:05.618243 (+ 32us) server_negotiation.cc:407] Connection header received
0811 14:35:05.618367 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:05.618375 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:05.618440 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:05.618600 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:05.619648 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:05.621050 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:05.621895 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:05.622361 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:05.622490 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:05.625060 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:05.625071 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:05.625072 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:05.625085 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:05.627520 (+ 2435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:05.627902 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:05.627912 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:05.627913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:05.628027 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:05.628257 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:05.628258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:05.628258 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:05.628538 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:05.628747 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:05.628848 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 14:35:05.629001 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":85,"threads_started":1}
I20260811 14:35:05.630307 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:05.633862 17575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:05.634583 17575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:05.639809 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:05.631842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53455 (local address 127.23.186.254:41313)
0811 14:35:05.631882 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:05.631884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:05.632935 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 14:35:05.633012 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:05.633013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:05.633025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:05.633090 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:05.633842 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:05.634411 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:05.635387 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:05.635612 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:05.635693 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:05.637089 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:05.637090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:05.637091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:05.637099 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:05.638588 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:05.638960 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:05.638962 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:05.638962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:05.639026 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:05.639243 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:05.639244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:05.639244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:05.639364 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:05.639503 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:05.639594 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 14:35:05.639692 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:05.665079 17575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2cf4bc39bc574178b84ffc195c9b8eef: Generated new TSK 0
I20260811 14:35:05.665328 17575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:05.670744 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17487
2026-08-11T14:35:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:05 dist-test-slave-cbsf krb5kdc[17597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:05 dist-test-slave-cbsf krb5kdc[17597](info): set up 2 sockets
Aug 11 14:35:05 dist-test-slave-cbsf krb5kdc[17597](info): commencing operation
krb5kdc: starting...
W20260811 14:35:06.802866 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:06 dist-test-slave-cbsf krb5kdc[17597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:06.935407 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:46263
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:44331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:46263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:07.023648 17612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:07.023895 17612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:07.023921 17612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:07.025089 17612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:07.025130 17612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:07.025143 17612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:07.025156 17612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:07.025168 17612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:07.026702 17612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:44331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:46263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:46263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:07.026925 17612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:07.027112 17612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:07.032305 17618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:07.032310 17620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:07.032305 17624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:07.032629 17612 server_base.cc:1034] running on GCE node
I20260811 14:35:07.032938 17612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:07.033267 17612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:07.034433 17612 hybrid_clock.cc:648] HybridClock initialized: now 1786458907034413 us; error 42 us; skew 500 ppm
Aug 11 14:35:07 dist-test-slave-cbsf krb5kdc[17597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458907, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:07.037211 17612 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:07.037732 17612 webserver.cc:460] Webserver started at http://127.23.186.254:40885/ using document root <none> and password file <none>
I20260811 14:35:07.037971 17612 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:07.038050 17612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:07.038208 17612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:07.039441 17612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "84053d772db9494382231f78da60f7f1"
format_stamp: "Formatted at 2026-08-11 14:35:07 on dist-test-slave-cbsf"
server_key: "8964ed7078afbf3427f6f9d2a5b539ec"
server_key_iv: "d4c022bf04e9736ace145a1d782ca69c"
server_key_version: "encryptionkey@0"
I20260811 14:35:07.039824 17612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "84053d772db9494382231f78da60f7f1"
format_stamp: "Formatted at 2026-08-11 14:35:07 on dist-test-slave-cbsf"
server_key: "8964ed7078afbf3427f6f9d2a5b539ec"
server_key_iv: "d4c022bf04e9736ace145a1d782ca69c"
server_key_version: "encryptionkey@0"
I20260811 14:35:07.041392 17612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:07.042186 17631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:07.042375 17612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:07.042487 17612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "84053d772db9494382231f78da60f7f1"
format_stamp: "Formatted at 2026-08-11 14:35:07 on dist-test-slave-cbsf"
server_key: "8964ed7078afbf3427f6f9d2a5b539ec"
server_key_iv: "d4c022bf04e9736ace145a1d782ca69c"
server_key_version: "encryptionkey@0"
I20260811 14:35:07.042565 17612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:07.056193 17612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:07.058708 17612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:07.058885 17612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:07.063388 17612 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:46263
I20260811 14:35:07.063436 17689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:46263 every 8 connection(s)
I20260811 14:35:07.063832 17612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:07.064756 17690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:07.067121 17690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1: Bootstrap starting.
I20260811 14:35:07.067718 17690 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:07.068019 17690 log.cc:824] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:07.068866 17690 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1: No bootstrap required, opened a new log
I20260811 14:35:07.070456 17690 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84053d772db9494382231f78da60f7f1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46263 } }
I20260811 14:35:07.070595 17690 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:07.070617 17690 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84053d772db9494382231f78da60f7f1, State: Initialized, Role: FOLLOWER
I20260811 14:35:07.070781 17690 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84053d772db9494382231f78da60f7f1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46263 } }
I20260811 14:35:07.070842 17690 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:07.070899 17690 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:07.070963 17690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:07.071835 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17612
I20260811 14:35:07.071961 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:07.071985 17690 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84053d772db9494382231f78da60f7f1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46263 } }
I20260811 14:35:07.072103 17690 leader_election.cc:302] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84053d772db9494382231f78da60f7f1; no voters:
I20260811 14:35:07.072150 24299 external_mini_cluster.cc:1428] Setting key a34ec75a5285951e0ddcd3f88f9f13c6
I20260811 14:35:07.072275 17690 leader_election.cc:288] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:07.072319 17693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:07.072504 17693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [term 1 LEADER]: Becoming Leader. State: Replica: 84053d772db9494382231f78da60f7f1, State: Running, Role: LEADER
I20260811 14:35:07.072543 17690 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:07.072628 17693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84053d772db9494382231f78da60f7f1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46263 } }
I20260811 14:35:07.073040 17693 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84053d772db9494382231f78da60f7f1. Latest consensus state: current_term: 1 leader_uuid: "84053d772db9494382231f78da60f7f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84053d772db9494382231f78da60f7f1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46263 } } }
I20260811 14:35:07.073108 17693 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:07.073213 17695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84053d772db9494382231f78da60f7f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84053d772db9494382231f78da60f7f1" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 46263 } } }
I20260811 14:35:07.073307 17702 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:07.073370 17695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:07.073907 17702 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:07.075599 17702 catalog_manager.cc:1347] Generated new cluster ID: 29ad8adceaf24e8e882a1d94fe262c1b
I20260811 14:35:07.075647 17702 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:07.078982 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:07.072862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36583 (local address 127.23.186.254:46263)
0811 14:35:07.073544 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:07.073554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:07.074423 (+ 869us) server_negotiation.cc:407] Connection header received
0811 14:35:07.074620 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:07.074626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:07.074706 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:07.074845 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:07.075097 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:07.076101 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:07.077009 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:07.077465 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:07.077587 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:07.078302 (+ 715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:07.078306 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:07.078307 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:07.078491 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:07.078523 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:07.078534 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:07.078612 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:07.078687 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 14:35:07.078754 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":85,"threads_started":1}
I20260811 14:35:07.079985 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:07.085274 17702 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:07.086098 17702 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:07.088160 17698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:07.081466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52561 (local address 127.23.186.254:46263)
0811 14:35:07.081507 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:07.081508 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:07.085242 (+ 3734us) server_negotiation.cc:407] Connection header received
0811 14:35:07.085378 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:07.085379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:07.085402 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:07.085447 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:07.085624 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:07.086113 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:07.086859 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:07.087088 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:07.087165 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:07.087878 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:07.087879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:07.087879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:07.087885 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:07.087900 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:07.087904 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:07.087961 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:07.088014 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 14:35:07.088059 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:07.099113 17702 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84053d772db9494382231f78da60f7f1: Generated new TSK 0
I20260811 14:35:07.099359 17702 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:07.104558 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17612
2026-08-11T14:35:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1435 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:07 dist-test-slave-cbsf krb5kdc[17724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:07 dist-test-slave-cbsf krb5kdc[17724](info): set up 2 sockets
Aug 11 14:35:07 dist-test-slave-cbsf krb5kdc[17724](info): commencing operation
krb5kdc: starting...
W20260811 14:35:08.223976 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:08 dist-test-slave-cbsf krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:08.355770 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:42789
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:42789
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:08.440001 17738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:08.440276 17738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:08.440317 17738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:08.441531 17738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:08.441606 17738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:08.441634 17738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:08.441668 17738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:08.441691 17738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:08.443305 17738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:42789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:42789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:08.443619 17738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:08.443868 17738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:08.449033 17746 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:08.449033 17744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:08.449033 17743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:08.449079 17738 server_base.cc:1034] running on GCE node
I20260811 14:35:08.449470 17738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:08.449783 17738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:08.450941 17738 hybrid_clock.cc:648] HybridClock initialized: now 1786458908450912 us; error 29 us; skew 500 ppm
Aug 11 14:35:08 dist-test-slave-cbsf krb5kdc[17724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458908, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:08.453670 17738 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:08.454149 17738 webserver.cc:460] Webserver started at http://127.23.186.254:46165/ using document root <none> and password file <none>
I20260811 14:35:08.454411 17738 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:08.454481 17738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:08.454641 17738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:08.455755 17738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "c017087a4a454ebcbc5e18b7477780ff"
format_stamp: "Formatted at 2026-08-11 14:35:08 on dist-test-slave-cbsf"
server_key: "b795a7649882d3149575f0f49252a5a9"
server_key_iv: "32bbf1f791a9979c9c622e8fc74ad379"
server_key_version: "encryptionkey@0"
I20260811 14:35:08.456136 17738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "c017087a4a454ebcbc5e18b7477780ff"
format_stamp: "Formatted at 2026-08-11 14:35:08 on dist-test-slave-cbsf"
server_key: "b795a7649882d3149575f0f49252a5a9"
server_key_iv: "32bbf1f791a9979c9c622e8fc74ad379"
server_key_version: "encryptionkey@0"
I20260811 14:35:08.457708 17738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 14:35:08.458485 17753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:08.458709 17738 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:08.458820 17738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "c017087a4a454ebcbc5e18b7477780ff"
format_stamp: "Formatted at 2026-08-11 14:35:08 on dist-test-slave-cbsf"
server_key: "b795a7649882d3149575f0f49252a5a9"
server_key_iv: "32bbf1f791a9979c9c622e8fc74ad379"
server_key_version: "encryptionkey@0"
I20260811 14:35:08.458897 17738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:08.462600 17738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:08.464841 17738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:08.465013 17738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:08.469272 17738 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:42789
I20260811 14:35:08.469306 17812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:42789 every 8 connection(s)
I20260811 14:35:08.469656 17738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:08.470736 17813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:08.471594 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17738
I20260811 14:35:08.471720 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:08.471935 24299 external_mini_cluster.cc:1428] Setting key 9dbf8d4eb2a8f93ebf5fdadeb8788f83
I20260811 14:35:08.473619 17813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff: Bootstrap starting.
I20260811 14:35:08.474432 17813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:08.474793 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:08.472535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42613 (local address 127.23.186.254:42789)
0811 14:35:08.472765 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:08.472773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:08.472800 (+ 27us) server_negotiation.cc:407] Connection header received
0811 14:35:08.473359 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:08.473365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:08.473391 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:08.473516 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:08.474369 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:08.474373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:08.474374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:08.474377 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:08.474417 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:08.474427 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:08.474534 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:08.474583 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 14:35:08.474646 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
I20260811 14:35:08.474808 17813 log.cc:824] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:08.475765 17813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff: No bootstrap required, opened a new log
I20260811 14:35:08.476116 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:08.478298 17813 raft_consensus.cc:348] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c017087a4a454ebcbc5e18b7477780ff" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42789 } }
I20260811 14:35:08.478502 17813 raft_consensus.cc:374] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 14:35:08.478561 17813 raft_consensus.cc:729] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c017087a4a454ebcbc5e18b7477780ff, State: Initialized, Role: FOLLOWER
I20260811 14:35:08.478701 17813 consensus_queue.cc:260] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c017087a4a454ebcbc5e18b7477780ff" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42789 } }
I20260811 14:35:08.478775 17816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:08.477661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45183 (local address 127.23.186.254:42789)
0811 14:35:08.477726 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:08.477729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:08.477887 (+ 158us) server_negotiation.cc:407] Connection header received
0811 14:35:08.477958 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:08.477958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:08.477963 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:08.478021 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:08.478558 (+ 537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:08.478559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:08.478559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 14:35:08.478563 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:08.478579 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 14:35:08.478583 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:08.478640 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:08.478705 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 14:35:08.478721 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 14:35:08.478803 17813 raft_consensus.cc:388] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 14:35:08.478840 17813 raft_consensus.cc:482] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 14:35:08.478881 17813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:08.479466 17813 raft_consensus.cc:504] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c017087a4a454ebcbc5e18b7477780ff" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42789 } }
I20260811 14:35:08.479557 17813 leader_election.cc:302] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c017087a4a454ebcbc5e18b7477780ff; no voters:
I20260811 14:35:08.479681 17813 leader_election.cc:288] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 14:35:08.479753 17823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:08.479926 17823 raft_consensus.cc:686] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [term 1 LEADER]: Becoming Leader. State: Replica: c017087a4a454ebcbc5e18b7477780ff, State: Running, Role: LEADER
I20260811 14:35:08.479954 17813 sys_catalog.cc:564] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:08.480036 17823 consensus_queue.cc:237] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c017087a4a454ebcbc5e18b7477780ff" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42789 } }
I20260811 14:35:08.480486 17825 sys_catalog.cc:455] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader c017087a4a454ebcbc5e18b7477780ff. Latest consensus state: current_term: 1 leader_uuid: "c017087a4a454ebcbc5e18b7477780ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c017087a4a454ebcbc5e18b7477780ff" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42789 } } }
I20260811 14:35:08.480595 17825 sys_catalog.cc:458] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:08.480800 17824 sys_catalog.cc:455] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c017087a4a454ebcbc5e18b7477780ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c017087a4a454ebcbc5e18b7477780ff" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 42789 } } }
I20260811 14:35:08.481328 17833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:08.481833 17833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:08.480903 17824 sys_catalog.cc:458] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:08.483565 17833 catalog_manager.cc:1347] Generated new cluster ID: ff2ac35d1df14606b89530f469b36ea7
I20260811 14:35:08.483620 17833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:08.491415 17833 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:08.491595 17833 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:08.502367 17833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c017087a4a454ebcbc5e18b7477780ff: Generated new TSK 0
I20260811 14:35:08.502601 17833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 14:35:08.511445 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17738
2026-08-11T14:35:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1406 ms)
[----------] 18 tests from AuthTokenIssuingTest (25764 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:08 dist-test-slave-cbsf krb5kdc[17849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:08 dist-test-slave-cbsf krb5kdc[17849](info): set up 2 sockets
Aug 11 14:35:08 dist-test-slave-cbsf krb5kdc[17849](info): commencing operation
krb5kdc: starting...
W20260811 14:35:09.641075 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:09 dist-test-slave-cbsf krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:09.772032 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40125
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:40125,127.23.186.253:46235,127.23.186.252:42667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:09.857282 17863 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:35:09.857573 17863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:09.857614 17863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:09.857658 17863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:09.858923 17863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:09.859002 17863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:09.859026 17863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:09.859050 17863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:09.859078 17863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:09.860734 17863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:40125,127.23.186.253:46235,127.23.186.252:42667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:40125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:09.861053 17863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:09.861306 17863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:09.866648 17868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:09.866690 17863 server_base.cc:1034] running on GCE node
W20260811 14:35:09.866654 17869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:09.866667 17871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:09.867136 17863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:09.867470 17863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:09.868678 17863 hybrid_clock.cc:648] HybridClock initialized: now 1786458909868655 us; error 42 us; skew 500 ppm
Aug 11 14:35:09 dist-test-slave-cbsf krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:09.871641 17863 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:09.872232 17863 webserver.cc:460] Webserver started at http://127.23.186.254:46515/ using document root <none> and password file <none>
I20260811 14:35:09.872450 17863 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:09.872498 17863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:09.872604 17863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:09.873879 17863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "fa54eb6e8fb14875b321c9011ebde32b"
format_stamp: "Formatted at 2026-08-11 14:35:09 on dist-test-slave-cbsf"
server_key: "c0a268b28dbd593bc3ee41c0566489a1"
server_key_iv: "10f704d109191d6fc94ea66d48d9c7fa"
server_key_version: "encryptionkey@0"
I20260811 14:35:09.874255 17863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "fa54eb6e8fb14875b321c9011ebde32b"
format_stamp: "Formatted at 2026-08-11 14:35:09 on dist-test-slave-cbsf"
server_key: "c0a268b28dbd593bc3ee41c0566489a1"
server_key_iv: "10f704d109191d6fc94ea66d48d9c7fa"
server_key_version: "encryptionkey@0"
I20260811 14:35:09.875842 17863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:09.876801 17878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:09.877027 17863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:09.877152 17863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "fa54eb6e8fb14875b321c9011ebde32b"
format_stamp: "Formatted at 2026-08-11 14:35:09 on dist-test-slave-cbsf"
server_key: "c0a268b28dbd593bc3ee41c0566489a1"
server_key_iv: "10f704d109191d6fc94ea66d48d9c7fa"
server_key_version: "encryptionkey@0"
I20260811 14:35:09.877224 17863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:09.899286 17863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:09.902045 17863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:09.902186 17863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:09.906595 17863 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:40125
I20260811 14:35:09.906780 17937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:40125 every 8 connection(s)
I20260811 14:35:09.907089 17863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:09.908133 17940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:09.908365 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17863
I20260811 14:35:09.908510 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:09.908767 24299 external_mini_cluster.cc:1428] Setting key ea884298a7977311e9c46bea7c4ea38b
I20260811 14:35:09.910024 17940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:09 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.186.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.253@KRBTEST.COM" created.
I20260811 14:35:09.921828 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:09.910843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40125 (local address 127.0.0.1:45746)
0811 14:35:09.911101 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:09.911141 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:09.911663 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:09.911992 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:09.911994 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:09.912274 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:09.912494 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:09.912498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:09.914153 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:09.914154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:09.914936 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:09.914938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:09.915696 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:09.915697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:09.915762 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:09.916375 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:09.916384 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:09.918222 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:09.920034 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:09.920041 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:09.920043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:09.920253 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:09.920543 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:09.920544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:09.920545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:09.920671 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:09.921149 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:09.921156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:09.921236 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 14:35:09.921341 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 14:35:09.921615 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":108,"thread_start_us":114,"threads_started":1}
I20260811 14:35:09.921836 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:09.910918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45746 (local address 127.23.186.254:40125)
0811 14:35:09.911096 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:09.911106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:09.911248 (+ 142us) server_negotiation.cc:407] Connection header received
0811 14:35:09.911831 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:09.911834 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:09.911855 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:09.911958 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:09.912607 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:09.914048 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:09.915032 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:09.915591 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:09.915722 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:09.918319 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:09.918329 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:09.918330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:09.918338 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:09.919943 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:09.920335 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:09.920338 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:09.920338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:09.920418 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:09.920796 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:09.920796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:09.920797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:09.921049 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:09.921163 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:09.921378 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 14:35:09.921537 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":75,"threads_started":1}
I20260811 14:35:09.922712 17940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:09.923090 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:09.922965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53480)
0811 14:35:09.923016 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:09.923066 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:35:09.923261 17880 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
I20260811 14:35:09.923760 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:09.923647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53496)
0811 14:35:09.923698 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:09.923731 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:35:09.923952 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:46235: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab.
Entry for principal kudu/127.23.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab.
I20260811 14:35:09.940059 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:46235
--webserver_interface=127.23.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab
--principal=kudu/127.23.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:40125,127.23.186.253:46235,127.23.186.252:42667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:35:09.973208 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } attempt: 1
I20260811 14:35:09.973834 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:09.973668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53504)
0811 14:35:09.973747 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:09.973798 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 14:35:09.974238 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:09.974140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53516)
0811 14:35:09.974187 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:09.974217 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 14:35:09.974376 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:46235: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
W20260811 14:35:10.026067 17947 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:35:10.026305 17947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:10.026324 17947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:10.026345 17947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:10.027558 17947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:10.027608 17947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:10.027623 17947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:10.027637 17947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:10.027649 17947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:10.029278 17947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:40125,127.23.186.253:46235,127.23.186.252:42667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab
--principal=kudu/127.23.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:46235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:10.029564 17947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:10.029820 17947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:10.035176 17956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:10.035109 17953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:10.035200 17954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:10.035534 17947 server_base.cc:1034] running on GCE node
I20260811 14:35:10.035812 17947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:10.036139 17947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:10.037312 17947 hybrid_clock.cc:648] HybridClock initialized: now 1786458910037296 us; error 45 us; skew 500 ppm
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:10.040181 17947 init.cc:377] Logged in from keytab as kudu/127.23.186.253@KRBTEST.COM (short username kudu)
I20260811 14:35:10.040735 17947 webserver.cc:460] Webserver started at http://127.23.186.253:34707/ using document root <none> and password file <none>
I20260811 14:35:10.041015 17947 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:10.041090 17947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:10.041241 17947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:10.042382 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data/instance:
uuid: "cbfd694a2134451bae1dee653b743c84"
format_stamp: "Formatted at 2026-08-11 14:35:10 on dist-test-slave-cbsf"
server_key: "a18bcb504ef2db91108ad92c76184082"
server_key_iv: "a32dd57959f3c222b2747534376aa680"
server_key_version: "encryptionkey@0"
I20260811 14:35:10.042569 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } attempt: 2
I20260811 14:35:10.042804 17947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal/instance:
uuid: "cbfd694a2134451bae1dee653b743c84"
format_stamp: "Formatted at 2026-08-11 14:35:10 on dist-test-slave-cbsf"
server_key: "a18bcb504ef2db91108ad92c76184082"
server_key_iv: "a32dd57959f3c222b2747534376aa680"
server_key_version: "encryptionkey@0"
I20260811 14:35:10.043005 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.042842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53520)
0811 14:35:10.042924 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.042973 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 14:35:10.043421 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.043294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53532)
0811 14:35:10.043351 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.043384 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:35:10.043550 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:46235: Network error: Client connection negotiation failed: client connection to 127.23.186.253:46235: connect: Connection refused (error 111)
I20260811 14:35:10.044446 17947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:10.045243 17963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:10.045421 17947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:10.045527 17947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal
uuid: "cbfd694a2134451bae1dee653b743c84"
format_stamp: "Formatted at 2026-08-11 14:35:10 on dist-test-slave-cbsf"
server_key: "a18bcb504ef2db91108ad92c76184082"
server_key_iv: "a32dd57959f3c222b2747534376aa680"
server_key_version: "encryptionkey@0"
I20260811 14:35:10.045615 17947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:10.071112 17947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:10.073210 17947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:10.073385 17947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:10.077636 17947 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.253:46235
I20260811 14:35:10.077692 18020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.253:46235 every 8 connection(s)
I20260811 14:35:10.078088 17947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
I20260811 14:35:10.079093 18021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:10.080899 18021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:10.086386 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 17947
I20260811 14:35:10.086517 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-1/wal/instance
I20260811 14:35:10.086702 24299 external_mini_cluster.cc:1428] Setting key 8ba1e17a64d8f1bb3aa0f3065c326aa8
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:10.091477 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.081660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45756 (local address 127.23.186.254:40125)
0811 14:35:10.081743 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.081745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.081932 (+ 187us) server_negotiation.cc:407] Connection header received
0811 14:35:10.082403 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.082404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.082417 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.082471 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.083356 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.084131 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.085245 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.085481 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.085571 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.088090 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.088091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.088092 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.088100 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.089944 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.090386 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.090389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.090389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.090458 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.090705 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.090706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.090707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.090871 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.090942 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.091220 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.091324 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 14:35:10.091502 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.081607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40125 (local address 127.0.0.1:45756)
0811 14:35:10.081807 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.081846 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.082319 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.082537 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.082541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.082894 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.083236 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.083242 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.084232 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.084233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.085151 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.085154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.085582 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.085583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.085716 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.086284 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.086295 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.087943 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.090059 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.090067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.090069 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.090309 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.090537 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.090538 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.090538 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.090640 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.090944 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.090950 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.091086 (+ 136us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.091177 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.091307 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":108,"threads_started":1}
I20260811 14:35:10.092259 18021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.23.186.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.252@KRBTEST.COM" created.
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:35:10.104043 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.092565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53546)
0811 14:35:10.092618 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.092629 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.092670 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.092964 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.092965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.093144 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.093218 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.093221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.094115 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.094116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.094953 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.094956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.095490 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.095492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.095575 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.095992 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.095998 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.100736 (+ 4738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.102846 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.102847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.102847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.103020 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.103253 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.103254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.103254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.103314 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.103649 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.103651 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.103693 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.103782 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.103916 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 14:35:10.104117 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.092660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53546 (local address 127.23.186.253:46235)
0811 14:35:10.092828 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.092832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.092848 (+ 16us) server_negotiation.cc:407] Connection header received
0811 14:35:10.092865 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.092867 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.092877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.092931 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.093306 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.094018 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.095066 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.095394 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.095460 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.100864 (+ 5404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.100879 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.100879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.100888 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.102735 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.103119 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.103123 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.103124 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.103183 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.103379 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.103379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.103380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.103551 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.103672 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.103818 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.103982 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":86,"server-negotiator.queue_time_us":144,"thread_start_us":117,"threads_started":1}
I20260811 14:35:10.105389 18021 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:10.105809 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.105690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40196)
0811 14:35:10.105741 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.105789 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 14:35:10.105949 17965 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
I20260811 14:35:10.106441 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.106342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40202)
0811 14:35:10.106391 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.106417 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:35:10.106544 18021 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:42667: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab.
Entry for principal kudu/127.23.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab.
I20260811 14:35:10.119917 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:42667
--webserver_interface=127.23.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab
--principal=kudu/127.23.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:33807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:40125,127.23.186.253:46235,127.23.186.252:42667
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:35:10.134693 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } attempt: 3
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:35:10.144191 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.135112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53548)
0811 14:35:10.135182 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.135194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.135264 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.135438 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.135440 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.135662 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.135755 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.135757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.136445 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.136446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.137484 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.137488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.137901 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.137903 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.137995 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.138498 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.138504 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.140083 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.142451 (+ 2368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.142452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.142453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.142718 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.143001 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.143002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.143002 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.143080 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.143890 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.143891 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.143943 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.144011 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.144111 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 14:35:10.144259 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.135190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53548 (local address 127.23.186.253:46235)
0811 14:35:10.135262 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.135264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.135275 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:10.135329 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.135330 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.135342 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.135399 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.135861 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.136364 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.137585 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.137810 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.137871 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.140195 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.140201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.140201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.140209 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.142326 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.142812 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.142814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.142815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.142904 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.143155 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.143156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.143157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.143813 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.143914 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.144061 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.144160 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 14:35:10.144889 17940 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:10.145315 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.145217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40206)
0811 14:35:10.145254 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.145295 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260811 14:35:10.145690 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.145591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40212)
0811 14:35:10.145642 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.145667 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 14:35:10.145810 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:42667: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
I20260811 14:35:10.155757 18021 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } attempt: 1
I20260811 14:35:10.156267 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.156123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40218)
0811 14:35:10.156192 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.156237 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:35:10.156623 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.156530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40226)
0811 14:35:10.156577 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.156601 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:35:10.156777 18021 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:42667: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
I20260811 14:35:10.176954 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } attempt: 1
I20260811 14:35:10.177467 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.177329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40228)
0811 14:35:10.177395 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.177436 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 14:35:10.177851 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.177752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40238)
0811 14:35:10.177800 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.177825 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:35:10.177954 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:42667: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
W20260811 14:35:10.214843 18028 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:35:10.215152 18028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:10.215183 18028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:10.215219 18028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:10.216501 18028 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:10.216583 18028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:10.216605 18028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:10.216626 18028 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:10.216655 18028 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:10.218314 18028 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:33807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:40125,127.23.186.253:46235,127.23.186.252:42667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab
--principal=kudu/127.23.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:42667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:10.218640 18028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:10.218935 18028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:10.224453 18036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:10.224673 18028 server_base.cc:1034] running on GCE node
W20260811 14:35:10.224548 18034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:10.224457 18033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:10.224911 18021 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } attempt: 2
I20260811 14:35:10.225090 18028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:10.225351 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.225227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40254)
0811 14:35:10.225280 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.225311 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 14:35:10.225510 18028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:10.225716 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.225595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40266)
0811 14:35:10.225641 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.225682 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260811 14:35:10.225824 18021 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:42667: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
I20260811 14:35:10.226683 18028 hybrid_clock.cc:648] HybridClock initialized: now 1786458910226669 us; error 38 us; skew 500 ppm
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:10.229388 18028 init.cc:377] Logged in from keytab as kudu/127.23.186.252@KRBTEST.COM (short username kudu)
I20260811 14:35:10.229852 18028 webserver.cc:460] Webserver started at http://127.23.186.252:33067/ using document root <none> and password file <none>
I20260811 14:35:10.230077 18028 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:10.230120 18028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:10.230230 18028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:10.231331 18028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data/instance:
uuid: "8e31464baa2a41268f261f22f9ebdeba"
format_stamp: "Formatted at 2026-08-11 14:35:10 on dist-test-slave-cbsf"
server_key: "ffd077d7802715e9e89e3b12607453a3"
server_key_iv: "215419c26b71b53d41ff031ebac5b5fc"
server_key_version: "encryptionkey@0"
I20260811 14:35:10.231649 18028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal/instance:
uuid: "8e31464baa2a41268f261f22f9ebdeba"
format_stamp: "Formatted at 2026-08-11 14:35:10 on dist-test-slave-cbsf"
server_key: "ffd077d7802715e9e89e3b12607453a3"
server_key_iv: "215419c26b71b53d41ff031ebac5b5fc"
server_key_version: "encryptionkey@0"
I20260811 14:35:10.233098 18028 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:10.233909 18044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:10.234123 18028 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 14:35:10.234264 18028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal
uuid: "8e31464baa2a41268f261f22f9ebdeba"
format_stamp: "Formatted at 2026-08-11 14:35:10 on dist-test-slave-cbsf"
server_key: "ffd077d7802715e9e89e3b12607453a3"
server_key_iv: "215419c26b71b53d41ff031ebac5b5fc"
server_key_version: "encryptionkey@0"
I20260811 14:35:10.234355 18028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:10.247694 18028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:10.250084 18028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:10.250269 18028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:10.253121 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } attempt: 2
I20260811 14:35:10.253707 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.253516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40282)
0811 14:35:10.253604 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.253659 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 14:35:10.254238 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.254093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40290)
0811 14:35:10.254150 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.254191 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 14:35:10.254350 17940 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:42667: Network error: Client connection negotiation failed: client connection to 127.23.186.252:42667: connect: Connection refused (error 111)
I20260811 14:35:10.254943 18028 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.252:42667
I20260811 14:35:10.254976 18105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.252:42667 every 8 connection(s)
I20260811 14:35:10.255385 18028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
I20260811 14:35:10.256410 18107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:10.256606 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 18028
I20260811 14:35:10.256757 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786458616937656-24299-0/minicluster-data/master-2/wal/instance
I20260811 14:35:10.256976 24299 external_mini_cluster.cc:1428] Setting key d5fa5dfdaa0d3fc3c2b411384a5e7989
I20260811 14:35:10.259111 18107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:10.268687 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.259923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45770 (local address 127.23.186.254:40125)
0811 14:35:10.259994 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.259996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.260201 (+ 205us) server_negotiation.cc:407] Connection header received
0811 14:35:10.260235 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.260236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.260251 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.260309 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.260872 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.261629 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.262727 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.263017 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.263087 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.265360 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.265361 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.265362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.265370 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.266899 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.267381 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.267382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.267382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.267452 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.267819 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.267821 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.267821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.268025 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.268111 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.268487 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.268592 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:35:10.268949 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.259839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:40125 (local address 127.0.0.1:45770)
0811 14:35:10.260065 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.260107 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.260186 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.260326 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.260329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.260676 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.260791 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.260793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.261744 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.261746 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.262636 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.262639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.263165 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.263166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.263234 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.263674 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.263683 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.265254 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.266990 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.266999 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.267002 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.267294 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.267523 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.267524 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.267525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.267676 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.268125 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.268132 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.268323 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.268445 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.268654 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":120,"threads_started":1}
I20260811 14:35:10.269575 18107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:10.269655 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.257595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.23.186.252:42667)
0811 14:35:10.257803 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.257814 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.257852 (+ 38us) server_negotiation.cc:407] Connection header received
0811 14:35:10.258335 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.258346 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.258424 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.258585 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.259266 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.260722 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.261856 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.262489 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.262621 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.265752 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.265762 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.265762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.265770 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.267721 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.268279 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.268282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.268282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.268349 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.268598 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.268598 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.268599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.268812 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.268950 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.269418 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.269510 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:10.277614 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.269919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:46235 (local address 127.0.0.1:53552)
0811 14:35:10.269954 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.269964 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.270004 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.270179 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.270180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.270350 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.270418 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.270420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.271364 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.271365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.272173 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.272175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.272660 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.272661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.272799 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.273266 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.273271 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.274681 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.276475 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.276477 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.276477 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.276742 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.276953 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.276954 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.276955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.277029 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.277300 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.277301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.277358 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.277428 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.277540 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260811 14:35:10.277709 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.270006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.23.186.253:46235)
0811 14:35:10.270052 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.270054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.270062 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:10.270086 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.270087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.270103 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.270154 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.270506 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.271275 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.272260 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.272574 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.272659 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.274780 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.274782 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.274782 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.274789 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.276371 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.276822 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.276823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.276824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.276882 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.277097 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.277097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.277098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.277233 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.277284 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.277506 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.277590 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:35:10.278350 18107 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:10.279304 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.270617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45786 (local address 127.23.186.254:40125)
0811 14:35:10.270653 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.270655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.270663 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:10.270709 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.270710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.270718 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.270759 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.271339 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.272059 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.272983 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.273188 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.273267 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.275536 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.275538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.275538 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.275545 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.277282 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.277683 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.277684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.277685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.277758 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.278199 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.278200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.278201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.278350 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.279135 (+ 785us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.279172 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.279241 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:35:10.286523 18111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.278668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:40304)
0811 14:35:10.278753 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.278764 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.278802 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.278905 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.278906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.279045 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.279074 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.279076 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.279894 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.279895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.280592 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.280595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.281104 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.281105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.281165 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.282240 (+ 1075us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.282245 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.283628 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.285292 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.285293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.285294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.285523 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.285734 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.285735 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.285736 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.285805 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.286110 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.286111 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.286166 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.286250 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.286364 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 14:35:10.286568 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.278721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.23.186.252:42667)
0811 14:35:10.278809 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.278811 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.278821 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:10.278850 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.278851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.278859 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.278905 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.279156 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.279822 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.280679 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.280994 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.281066 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.283716 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.283717 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.283718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.283725 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.285200 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.285606 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.285607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.285607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.285661 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.285873 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.285874 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.285874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.286036 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.286090 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.286279 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.286429 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":71}
I20260811 14:35:10.288004 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.280081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.23.186.253:46235)
0811 14:35:10.280120 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.280122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.280129 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:10.280150 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.280151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.280160 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.280225 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.280808 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.281442 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.282219 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.282425 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.282494 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.285050 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.285051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.285051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.285060 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.286757 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.287178 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.287179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.287180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.287240 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.287430 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.287431 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.287432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.287589 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.287787 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.287827 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.287904 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:10.288668 18107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: Bootstrap starting.
I20260811 14:35:10.290015 18107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:10.290097 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:10.290326 18107 log.cc:824] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:10.291177 18107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: No bootstrap required, opened a new log
I20260811 14:35:10.291113 17908 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fa54eb6e8fb14875b321c9011ebde32b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45786
I20260811 14:35:10.292850 18107 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.293009 18107 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e31464baa2a41268f261f22f9ebdeba, State: Initialized, Role: FOLLOWER
I20260811 14:35:10.293110 18107 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.293488 18107 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:10.293486 18113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.293610 18113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:35:10.295342 18126 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:10.295441 18126 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:10.295761 18028 master_runner.cc:186] Error getting master registration for 127.23.186.254:40125: OK, instance_id { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" instance_seqno: 1786458909905327 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:35:10.299067 18028 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:35:10.316987 18021 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } attempt: 3
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458910, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:10.325810 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.317511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.23.186.252:42667)
0811 14:35:10.317598 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.317600 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.317614 (+ 14us) server_negotiation.cc:407] Connection header received
0811 14:35:10.317661 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.317662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.317674 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.317737 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.318155 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.318948 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.320018 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.320318 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.320398 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.322599 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.322601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.322601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.322610 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.324318 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.324855 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.324857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.324857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.324925 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.325132 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.325133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.325133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.325349 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.325424 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.325621 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.325712 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 14:35:10.325807 18023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.317422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:51578)
0811 14:35:10.317502 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.317520 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.317593 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.317768 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.317769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.317995 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.318054 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.318057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.319055 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.319056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.319886 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.319889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.320407 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.320408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.320473 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.321044 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.321050 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.322490 (+ 1440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.324458 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.324459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.324460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.324759 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.324997 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.324998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.324998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.325067 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.325422 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.325423 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.325506 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.325578 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.325691 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 14:35:10.327759 18021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: Bootstrap starting.
I20260811 14:35:10.328565 18021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:10.328929 18021 log.cc:824] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:10.329818 18021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: No bootstrap required, opened a new log
I20260811 14:35:10.331919 18021 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.332096 18021 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfd694a2134451bae1dee653b743c84, State: Initialized, Role: FOLLOWER
I20260811 14:35:10.332190 18021 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.332567 18021 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:10.332512 18129 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.332644 18129 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:35:10.334707 18141 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:10.334831 18141 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:10.334945 17947 master_runner.cc:186] Error getting master registration for 127.23.186.254:40125: OK, instance_id { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" instance_seqno: 1786458909905327 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:35:10.338299 17947 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:35:10.353550 17940 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } attempt: 3
Aug 11 14:35:10 dist-test-slave-cbsf krb5kdc[17849](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458909, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:10.362533 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.353969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:42667 (local address 127.0.0.1:51588)
0811 14:35:10.354049 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:10.354063 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:10.354109 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:10.354334 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:10.354336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:10.354560 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:10.354621 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.354624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.355621 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.355622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.356470 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:10.356475 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.357006 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.357007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:10.357066 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.357637 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:10.357643 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:10.359380 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:10.361359 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.361360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.361361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.361605 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.361834 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:10.361835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:10.361835 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:10.361904 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:10.362210 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:10.362212 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:10.362266 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 14:35:10.362314 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 14:35:10.362427 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 14:35:10.362982 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.354062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.23.186.252:42667)
0811 14:35:10.354192 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.354194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.354203 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:10.354232 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.354233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.354244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.354298 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.354739 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.355522 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.356593 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.356929 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.357006 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.359510 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.359512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.359513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.359522 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.361267 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.361698 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.361700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.361700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.361771 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.361971 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.361972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.361972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.362131 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.362751 (+ 620us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.362794 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.362892 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 14:35:10.364662 17940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b: Bootstrap starting.
I20260811 14:35:10.365535 17940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:10.365839 17940 log.cc:824] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:10.366765 17940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b: No bootstrap required, opened a new log
I20260811 14:35:10.368212 17940 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.368381 17940 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa54eb6e8fb14875b321c9011ebde32b, State: Initialized, Role: FOLLOWER
I20260811 14:35:10.368466 17940 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.368793 18145 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.368953 17908 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 14:35:10.368894 17940 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:10.368969 18145 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:10.369010 17908 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:10.371285 17908 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.371560 17908 leader_election.cc:288] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [CANDIDATE]: Term 1 election: Requested vote from peers cbfd694a2134451bae1dee653b743c84 (127.23.186.253:46235), 8e31464baa2a41268f261f22f9ebdeba (127.23.186.252:42667)
I20260811 14:35:10.372260 17993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa54eb6e8fb14875b321c9011ebde32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cbfd694a2134451bae1dee653b743c84"
I20260811 14:35:10.372429 17993 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:10.373456 17993 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa54eb6e8fb14875b321c9011ebde32b in term 1.
I20260811 14:35:10.373697 18074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa54eb6e8fb14875b321c9011ebde32b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8e31464baa2a41268f261f22f9ebdeba"
I20260811 14:35:10.373823 18074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [term 0 FOLLOWER]: Advancing to term 1
W20260811 14:35:10.373826 18164 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:10.373874 18164 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:10.374655 18074 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa54eb6e8fb14875b321c9011ebde32b in term 1.
I20260811 14:35:10.375450 17880 leader_election.cc:302] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cbfd694a2134451bae1dee653b743c84, fa54eb6e8fb14875b321c9011ebde32b; no voters:
I20260811 14:35:10.375646 18145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:10.375787 18145 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [term 1 LEADER]: Becoming Leader. State: Replica: fa54eb6e8fb14875b321c9011ebde32b, State: Running, Role: LEADER
I20260811 14:35:10.375866 18145 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } }
I20260811 14:35:10.376307 18145 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa54eb6e8fb14875b321c9011ebde32b. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.376386 18145 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:10.378297 17863 master_runner.cc:263] No leader master found from master fa54eb6e8fb14875b321c9011ebde32b
I20260811 14:35:10.378346 17863 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:35:10.379065 18168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:10.379383 18168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:10.383524 18074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [term 1 FOLLOWER]: Refusing update from remote peer fa54eb6e8fb14875b321c9011ebde32b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:10.383951 18145 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:10.385524 18113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa54eb6e8fb14875b321c9011ebde32b. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.385622 18113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:10.386158 17993 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [term 1 FOLLOWER]: Refusing update from remote peer fa54eb6e8fb14875b321c9011ebde32b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:10.386497 18145 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:10.387885 18129 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa54eb6e8fb14875b321c9011ebde32b. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.387982 18129 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:10.397068 18113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.397205 18113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:10.397392 18174 mvcc.cc:205] Tried to move back new op lower bound from 7317335696923385856 to 7317335696900300800. Current Snapshot: MvccSnapshot[applied={T|T < 7317335696923385856}]
I20260811 14:35:10.396662 18166 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.397502 18166 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:10.397550 18168 catalog_manager.cc:1347] Generated new cluster ID: 63db83eaaf5a4e17b9ff554450f18ad2
I20260811 14:35:10.397583 18168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 14:35:10.399701 18172 mvcc.cc:205] Tried to move back new op lower bound from 7317335696923385856 to 7317335696900300800. Current Snapshot: MvccSnapshot[applied={T|T < 7317335696923385856 or (T in {7317335696923385856})}]
I20260811 14:35:10.402351 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.379848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49426 (local address 127.23.186.254:40125)
0811 14:35:10.379904 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.379906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.382103 (+ 2197us) server_negotiation.cc:407] Connection header received
0811 14:35:10.382214 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.382215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.382227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.382258 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.388204 (+ 5946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.388958 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.391118 (+ 2160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.391414 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.391466 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.395758 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.395759 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.395760 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.395768 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.398109 (+ 2341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.400555 (+ 2446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.400556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.400557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.400625 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.401172 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.401173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.401173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.401358 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.401458 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.402158 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.402256 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 14:35:10.403574 18025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.380080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46350 (local address 127.23.186.253:46235)
0811 14:35:10.380120 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.380122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.388871 (+ 8749us) server_negotiation.cc:407] Connection header received
0811 14:35:10.388983 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.388984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.388995 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.389035 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.389881 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.390609 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.391629 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.392264 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.392309 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.395462 (+ 3153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.395464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.395464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.395472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.399349 (+ 3877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.400877 (+ 1528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.400879 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.400879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.400940 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.401677 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.401679 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.401679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.401840 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.401924 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.403387 (+ 1463us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.403483 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260811 14:35:10.405934 18166 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.406052 18166 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:10.406693 18129 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa54eb6e8fb14875b321c9011ebde32b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa54eb6e8fb14875b321c9011ebde32b" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 40125 } } peers { permanent_uuid: "cbfd694a2134451bae1dee653b743c84" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 46235 } } peers { permanent_uuid: "8e31464baa2a41268f261f22f9ebdeba" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 42667 } } }
I20260811 14:35:10.406787 18129 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:10.407109 18176 mvcc.cc:205] Tried to move back new op lower bound from 7317335696923385856 to 7317335696900300800. Current Snapshot: MvccSnapshot[applied={T|T < 7317335696923385856}]
I20260811 14:35:10.407310 18168 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:10.408038 18168 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:10.411465 18110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:10.380683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.23.186.252:42667)
0811 14:35:10.380754 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.380757 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.390167 (+ 9410us) server_negotiation.cc:407] Connection header received
0811 14:35:10.390271 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.390272 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.390285 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.390316 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.391869 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.392566 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.394336 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:10.394643 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:10.394692 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:10.408089 (+ 13397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:10.408091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:10.408091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:10.408101 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:10.410082 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.410529 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.410531 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.410531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.410591 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:10.410807 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:10.410808 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:10.410808 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:10.410946 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:10.411048 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:10.411282 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 14:35:10.411371 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 14:35:10.417481 18168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa54eb6e8fb14875b321c9011ebde32b: Generated new TSK 0
I20260811 14:35:10.417748 18168 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:35:10.445832 17944 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.443953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.23.186.254:40125)
0811 14:35:10.444000 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.444002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.445054 (+ 1052us) server_negotiation.cc:407] Connection header received
0811 14:35:10.445174 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.445175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.445188 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.445223 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.445735 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49436: BlockingRecv error: recv got EOF from 127.0.0.1:49436 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 14:35:10.446811 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.444407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46358 (local address 127.23.186.253:46235)
0811 14:35:10.444449 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.444450 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.446280 (+ 1830us) server_negotiation.cc:407] Connection header received
0811 14:35:10.446381 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.446382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.446395 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.446426 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.446750 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46358: BlockingRecv error: recv got EOF from 127.0.0.1:46358 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 14:35:10.447873 18110 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.444755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.23.186.252:42667)
0811 14:35:10.444797 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.444799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.447158 (+ 2359us) server_negotiation.cc:407] Connection header received
0811 14:35:10.447287 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.447289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.447299 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.447328 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.447806 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51612: BlockingRecv error: recv got EOF from 127.0.0.1:51612 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 14:35:10.450968 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.450084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.23.186.253:46235)
0811 14:35:10.450125 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.450127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.450363 (+ 236us) server_negotiation.cc:407] Connection header received
0811 14:35:10.450482 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.450483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.450502 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.450535 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.450913 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46368: BlockingRecv error: recv got EOF from 127.0.0.1:46368 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 14:35:10.454453 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.453556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.23.186.253:46235)
0811 14:35:10.453594 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.453595 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.453866 (+ 271us) server_negotiation.cc:407] Connection header received
0811 14:35:10.453970 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.453970 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.453982 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.454014 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.454400 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46372: BlockingRecv error: recv got EOF from 127.0.0.1:46372 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:35:10.458719 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.457852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46388 (local address 127.23.186.253:46235)
0811 14:35:10.457891 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.457893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.458153 (+ 260us) server_negotiation.cc:407] Connection header received
0811 14:35:10.458256 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.458257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.458269 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.458299 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.458665 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46388: BlockingRecv error: recv got EOF from 127.0.0.1:46388 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:35:10.465224 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.464315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46402 (local address 127.23.186.253:46235)
0811 14:35:10.464358 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.464360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.464607 (+ 247us) server_negotiation.cc:407] Connection header received
0811 14:35:10.464735 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.464736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.464747 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.464779 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.465156 (+ 377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46402: BlockingRecv error: recv got EOF from 127.0.0.1:46402 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260811 14:35:10.475805 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.474864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46406 (local address 127.23.186.253:46235)
0811 14:35:10.474910 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.474913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.475196 (+ 283us) server_negotiation.cc:407] Connection header received
0811 14:35:10.475302 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.475303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.475314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.475345 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.475742 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46406: BlockingRecv error: recv got EOF from 127.0.0.1:46406 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 14:35:10.494318 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.493395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46418 (local address 127.23.186.253:46235)
0811 14:35:10.493445 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.493448 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.493651 (+ 203us) server_negotiation.cc:407] Connection header received
0811 14:35:10.493720 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.493721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.493734 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.493819 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.494241 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46418: BlockingRecv error: recv got EOF from 127.0.0.1:46418 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 14:35:10.528352 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.527524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46426 (local address 127.23.186.253:46235)
0811 14:35:10.527575 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.527578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.527732 (+ 154us) server_negotiation.cc:407] Connection header received
0811 14:35:10.527816 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.527817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.527830 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.527902 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.528276 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46426: BlockingRecv error: recv got EOF from 127.0.0.1:46426 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260811 14:35:10.594600 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.593729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46442 (local address 127.23.186.253:46235)
0811 14:35:10.593793 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.593795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.593902 (+ 107us) server_negotiation.cc:407] Connection header received
0811 14:35:10.593950 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.593951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.593962 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.594037 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.594515 (+ 478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46442: BlockingRecv error: recv got EOF from 127.0.0.1:46442 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 14:35:10.724578 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.723830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46456 (local address 127.23.186.253:46235)
0811 14:35:10.723933 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.723935 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.723949 (+ 14us) server_negotiation.cc:407] Connection header received
0811 14:35:10.724006 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.724007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.724018 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.724087 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.724504 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46456: BlockingRecv error: recv got EOF from 127.0.0.1:46456 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 14:35:10.982568 18025 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:10.981925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46462 (local address 127.23.186.253:46235)
0811 14:35:10.982023 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:10.982025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:10.982037 (+ 12us) server_negotiation.cc:407] Connection header received
0811 14:35:10.982069 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:10.982070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:10.982081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:10.982150 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:10.982517 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46462: BlockingRecv error: recv got EOF from 127.0.0.1:46462 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 14:35:11.295938 18126 catalog_manager.cc:1259] Loaded cluster ID: 63db83eaaf5a4e17b9ff554450f18ad2
I20260811 14:35:11.296029 18126 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: loading cluster ID for follower catalog manager: success
I20260811 14:35:11.296921 18126 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: acquiring CA information for follower catalog manager: success
I20260811 14:35:11.297281 18126 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e31464baa2a41268f261f22f9ebdeba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:35:11.335350 18141 catalog_manager.cc:1259] Loaded cluster ID: 63db83eaaf5a4e17b9ff554450f18ad2
I20260811 14:35:11.335479 18141 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: loading cluster ID for follower catalog manager: success
I20260811 14:35:11.336325 18141 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: acquiring CA information for follower catalog manager: success
I20260811 14:35:11.336668 18141 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cbfd694a2134451bae1dee653b743c84: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:35:11.498726 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:11.495871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46476 (local address 127.23.186.253:46235)
0811 14:35:11.495989 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:11.495992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:11.496028 (+ 36us) server_negotiation.cc:407] Connection header received
0811 14:35:11.496116 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:11.496117 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:11.496187 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:11.496273 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:11.496448 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.497011 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.497833 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.498052 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.498129 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:11.498330 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:11.498463 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:11.498567 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:11.498591 (+ 24us) server_negotiation.cc:299] Negotiation successful
0811 14:35:11.498616 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":56,"threads_started":1}
I20260811 14:35:11.503221 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:11.500768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51614 (local address 127.23.186.252:42667)
0811 14:35:11.500887 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:11.500889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:11.500913 (+ 24us) server_negotiation.cc:407] Connection header received
0811 14:35:11.500987 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:11.500987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:11.501029 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:11.501096 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:11.501171 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.501662 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.502468 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.502677 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.502733 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:11.502873 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:11.502985 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:11.503048 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:11.503085 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 14:35:11.503104 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":70,"threads_started":1}
I20260811 14:35:11.508350 18271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:11.505457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51622 (local address 127.23.186.252:42667)
0811 14:35:11.505501 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:11.505502 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:11.505848 (+ 346us) server_negotiation.cc:407] Connection header received
0811 14:35:11.505879 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:11.505880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:11.505891 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:11.505949 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:11.506091 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.506749 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.507551 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.507840 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.507881 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:11.508090 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:11.508124 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:11.508178 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:11.508231 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 14:35:11.508259 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:11.508421 18264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:11.505209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46490 (local address 127.23.186.253:46235)
0811 14:35:11.505434 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:11.505436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:11.505443 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:11.505473 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:11.505474 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:11.505485 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:11.505532 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:11.505748 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.506528 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.507532 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.507837 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.507901 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:11.508117 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:11.508152 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:11.508201 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:11.508292 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 14:35:11.508319 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260811 14:35:11.508510 18279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:11.505141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49446 (local address 127.23.186.254:40125)
0811 14:35:11.505253 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:11.505256 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:11.505266 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:11.505320 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:11.505321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:11.505389 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:11.505474 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:11.505620 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.506326 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.507398 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:11.507671 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:11.507765 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:11.507973 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:11.508108 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:11.508174 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:11.508392 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 14:35:11.508420 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":52,"threads_started":1}
I20260811 14:35:11.516168 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17863
I20260811 14:35:11.524292 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 17947
I20260811 14:35:11.531405 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 18028
2026-08-11T14:35:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3020 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 14:35:11 dist-test-slave-cbsf krb5kdc[18283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 14:35:11 dist-test-slave-cbsf krb5kdc[18283](info): set up 2 sockets
Aug 11 14:35:11 dist-test-slave-cbsf krb5kdc[18283](info): commencing operation
krb5kdc: starting...
W20260811 14:35:12.703245 24299 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/test-user.keytab.
Aug 11 14:35:12 dist-test-slave-cbsf krb5kdc[18283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T14:35:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T14:35:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.23.186.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
Entry for principal kudu/127.23.186.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab.
I20260811 14:35:12.839792 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44751
--webserver_interface=127.23.186.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:44751,127.23.186.253:36381,127.23.186.252:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 14:35:12.923388 18299 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:35:12.923658 18299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:12.923689 18299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:12.923735 18299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:12.924934 18299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:12.925004 18299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:12.925029 18299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:12.925064 18299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:12.925091 18299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:12.926620 18299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:44751,127.23.186.253:36381,127.23.186.252:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.254.keytab
--principal=kudu/127.23.186.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.254:44751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.23.186.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:12.926908 18299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:12.927158 18299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:12.932200 18306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:12.932266 18299 server_base.cc:1034] running on GCE node
W20260811 14:35:12.932205 18305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:12.932210 18309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:12.932610 18299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:12.932925 18299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:12.934092 18299 hybrid_clock.cc:648] HybridClock initialized: now 1786458912934081 us; error 45 us; skew 500 ppm
Aug 11 14:35:12 dist-test-slave-cbsf krb5kdc[18283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:12.936542 18299 init.cc:377] Logged in from keytab as kudu/127.23.186.254@KRBTEST.COM (short username kudu)
I20260811 14:35:12.937067 18299 webserver.cc:460] Webserver started at http://127.23.186.254:45089/ using document root <none> and password file <none>
I20260811 14:35:12.937336 18299 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:12.937405 18299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:12.937547 18299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:12.938607 18299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data/instance:
uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3"
format_stamp: "Formatted at 2026-08-11 14:35:12 on dist-test-slave-cbsf"
server_key: "a8f9597ff8a4e58c950ca9d91f1ab5df"
server_key_iv: "74fffd1ff7fee8731ecbbb9667180989"
server_key_version: "encryptionkey@0"
I20260811 14:35:12.938970 18299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal/instance:
uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3"
format_stamp: "Formatted at 2026-08-11 14:35:12 on dist-test-slave-cbsf"
server_key: "a8f9597ff8a4e58c950ca9d91f1ab5df"
server_key_iv: "74fffd1ff7fee8731ecbbb9667180989"
server_key_version: "encryptionkey@0"
I20260811 14:35:12.940301 18299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 14:35:12.941118 18316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:12.941282 18299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:12.941382 18299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal
uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3"
format_stamp: "Formatted at 2026-08-11 14:35:12 on dist-test-slave-cbsf"
server_key: "a8f9597ff8a4e58c950ca9d91f1ab5df"
server_key_iv: "74fffd1ff7fee8731ecbbb9667180989"
server_key_version: "encryptionkey@0"
I20260811 14:35:12.941470 18299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:12.949532 18299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:12.951574 18299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:12.951736 18299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:12.956049 18299 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.254:44751
I20260811 14:35:12.956082 18373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.254:44751 every 8 connection(s)
I20260811 14:35:12.956470 18299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/data/info.pb
I20260811 14:35:12.957417 18374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:12.958992 18374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:12 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:12.965977 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 18299
I20260811 14:35:12.966055 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-0/wal/instance
I20260811 14:35:12.966292 24299 external_mini_cluster.cc:1428] Setting key 82d37355d28ecfa6bf2683f335309ff5
I20260811 14:35:12.969465 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:12.959677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44751 (local address 127.0.0.1:33608)
0811 14:35:12.959917 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:12.959962 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:12.960480 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:12.960796 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:12.960799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:12.961080 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:12.961227 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:12.961231 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:12.962273 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:12.962274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:12.962945 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:12.962947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:12.963469 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:12.963471 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:12.963536 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:12.964058 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:12.964067 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:12.965550 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:12.967751 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:12.967775 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:12.967777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:12.968074 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:12.968353 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:12.968354 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:12.968354 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:12.968447 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:12.968971 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:12.968979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:12.969079 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 14:35:12.969164 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 14:35:12.969299 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260811 14:35:12.969548 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:12.959755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33608 (local address 127.23.186.254:44751)
0811 14:35:12.959930 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:12.959939 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:12.960066 (+ 127us) server_negotiation.cc:407] Connection header received
0811 14:35:12.960647 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:12.960649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:12.960666 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:12.960791 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:12.961330 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:12.962158 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:12.963045 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:12.963365 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:12.963496 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:12.965657 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:12.965664 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:12.965665 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:12.965672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:12.967651 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:12.968178 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:12.968180 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:12.968181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:12.968284 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:12.968514 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:12.968515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:12.968515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:12.968856 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:12.968949 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:12.969224 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 14:35:12.969392 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260811 14:35:12.971405 18374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:12.971791 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:12.971673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51054)
0811 14:35:12.971721 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:12.971768 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260811 14:35:12.971959 18321 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
I20260811 14:35:12.972404 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:12.972316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51070)
0811 14:35:12.972362 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:12.972387 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 14:35:12.972548 18374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:36381: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.23.186.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab.
Entry for principal kudu/127.23.186.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab.
I20260811 14:35:12.996723 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:36381
--webserver_interface=127.23.186.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab
--principal=kudu/127.23.186.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:44751,127.23.186.253:36381,127.23.186.252:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:35:13.021818 18374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } attempt: 1
I20260811 14:35:13.022358 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.022218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51082)
0811 14:35:13.022286 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.022327 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:35:13.022725 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.022612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51084)
0811 14:35:13.022661 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.022707 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 14:35:13.022874 18374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:36381: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
W20260811 14:35:13.084283 18382 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:35:13.084498 18382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:13.084517 18382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:13.084544 18382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:13.085738 18382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:13.085783 18382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:13.085798 18382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:13.085810 18382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:13.085822 18382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:13.087328 18382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:44751,127.23.186.253:36381,127.23.186.252:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.253.keytab
--principal=kudu/127.23.186.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.253:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.23.186.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:13.087600 18382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:13.087831 18382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 14:35:13.091048 18374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } attempt: 2
I20260811 14:35:13.091531 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.091388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51088)
0811 14:35:13.091456 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.091498 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 14:35:13.092361 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.092259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51100)
0811 14:35:13.092307 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.092333 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 14:35:13.092536 18374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.253:36381: Network error: Client connection negotiation failed: client connection to 127.23.186.253:36381: connect: Connection refused (error 111)
W20260811 14:35:13.092866 18388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:13.092906 18389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:13.093148 18391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:13.093304 18382 server_base.cc:1034] running on GCE node
I20260811 14:35:13.093536 18382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:13.093778 18382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:13.094949 18382 hybrid_clock.cc:648] HybridClock initialized: now 1786458913094934 us; error 28 us; skew 500 ppm
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:13.097594 18382 init.cc:377] Logged in from keytab as kudu/127.23.186.253@KRBTEST.COM (short username kudu)
I20260811 14:35:13.098060 18382 webserver.cc:460] Webserver started at http://127.23.186.253:36997/ using document root <none> and password file <none>
I20260811 14:35:13.098315 18382 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:13.098389 18382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:13.098546 18382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:13.099673 18382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data/instance:
uuid: "8a15426942c74ff1ae1ecaeba53ee87e"
format_stamp: "Formatted at 2026-08-11 14:35:13 on dist-test-slave-cbsf"
server_key: "b07d93226f42fadb69c8344dd026e31f"
server_key_iv: "6f9cf5dcb72a828611e9d38b8774d34a"
server_key_version: "encryptionkey@0"
I20260811 14:35:13.100052 18382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal/instance:
uuid: "8a15426942c74ff1ae1ecaeba53ee87e"
format_stamp: "Formatted at 2026-08-11 14:35:13 on dist-test-slave-cbsf"
server_key: "b07d93226f42fadb69c8344dd026e31f"
server_key_iv: "6f9cf5dcb72a828611e9d38b8774d34a"
server_key_version: "encryptionkey@0"
I20260811 14:35:13.101554 18382 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260811 14:35:13.102396 18401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:13.102584 18382 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 14:35:13.102696 18382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal
uuid: "8a15426942c74ff1ae1ecaeba53ee87e"
format_stamp: "Formatted at 2026-08-11 14:35:13 on dist-test-slave-cbsf"
server_key: "b07d93226f42fadb69c8344dd026e31f"
server_key_iv: "6f9cf5dcb72a828611e9d38b8774d34a"
server_key_version: "encryptionkey@0"
I20260811 14:35:13.102779 18382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:13.114642 18382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:13.117144 18382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:13.117318 18382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:13.121387 18382 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.253:36381
I20260811 14:35:13.121425 18458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.253:36381 every 8 connection(s)
I20260811 14:35:13.121826 18382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/data/info.pb
I20260811 14:35:13.122748 18459 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:13.124382 18459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:13.124675 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 18382
I20260811 14:35:13.124795 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-1/wal/instance
I20260811 14:35:13.125012 24299 external_mini_cluster.cc:1428] Setting key 9a57b9084568d0f143e21e67fa0cc935
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.23.186.252@KRBTEST.COM; defaulting to no policy
I20260811 14:35:13.134541 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.125294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.23.186.254:44751)
0811 14:35:13.125342 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.125345 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.125554 (+ 209us) server_negotiation.cc:407] Connection header received
0811 14:35:13.126110 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.126111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.126130 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.126186 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.126960 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.127456 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.128796 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.129020 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.129119 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.131392 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.131393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.131394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.131403 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.133136 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.133532 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.133533 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.133534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.133610 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.133901 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.133902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.133903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.134065 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.134144 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.134401 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.134472 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260811 14:35:13.134760 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.125251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44751 (local address 127.0.0.1:33616)
0811 14:35:13.125426 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.125464 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.125985 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.126256 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.126260 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.126526 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.126848 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.126854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.127559 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.127560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.128680 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.128683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.129130 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.129132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.129270 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.129881 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.129891 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.131286 (+ 1395us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.133220 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.133228 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.133230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.133453 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.133685 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.133686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.133686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.133833 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.134150 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.134157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.134281 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.134368 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.134492 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260811 14:35:13.135422 18459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.23.186.252@KRBTEST.COM" created.
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:35:13.143608 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.135718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51108)
0811 14:35:13.135772 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.135783 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.135832 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.136127 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.136128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.136272 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.136323 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.136324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.137347 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.137348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.138171 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.138174 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.138641 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.138642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.138723 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.139291 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.139295 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.140434 (+ 1139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.142432 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.142432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.142433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.142659 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.142852 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.142852 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.142853 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.142897 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.143275 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.143276 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.143315 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.143381 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.143490 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 14:35:13.143690 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.135784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51108 (local address 127.23.186.253:36381)
0811 14:35:13.135954 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.135959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.135977 (+ 18us) server_negotiation.cc:407] Connection header received
0811 14:35:13.136007 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.136009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.136020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.136128 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.136389 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.137245 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.138256 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.138560 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.138620 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.140532 (+ 1912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.140541 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.140541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.140548 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.142342 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.142739 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.142743 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.142743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.142794 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.142960 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.142960 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.142961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.143192 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.143299 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.143411 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.143547 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260811 14:35:13.144313 18459 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:13.144737 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.144564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47638)
0811 14:35:13.144624 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.144680 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:35:13.144910 18402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
I20260811 14:35:13.145380 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.145268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47644)
0811 14:35:13.145322 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.145353 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 14:35:13.145495 18459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:46639: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.23.186.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab.
Entry for principal kudu/127.23.186.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab.
I20260811 14:35:13.155308 24299 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLI13IK/build/release/bin/kudu
/tmp/dist-test-taskLI13IK/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:46639
--webserver_interface=127.23.186.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab
--principal=kudu/127.23.186.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.23.186.212:35459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.23.186.254:44751,127.23.186.253:36381,127.23.186.252:46639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 14:35:13.183727 18374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } attempt: 3
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:35:13.191128 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.184308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51120 (local address 127.23.186.253:36381)
0811 14:35:13.184377 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.184380 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.184410 (+ 30us) server_negotiation.cc:407] Connection header received
0811 14:35:13.184441 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.184446 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.184457 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.184522 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.184925 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.185444 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.186178 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.186408 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.186491 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.188434 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.188435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.188436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.188444 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.189883 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.190273 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.190275 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.190275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.190338 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.190558 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.190559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.190560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.190705 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.190778 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.190991 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.191060 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 14:35:13.191160 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.184178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51120)
0811 14:35:13.184265 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.184280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.184371 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.184527 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.184529 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.184750 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.184831 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.184833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.185537 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.185538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.186085 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.186087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.186488 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.186489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.186565 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.187089 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.187094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.188338 (+ 1244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.190013 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.190014 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.190015 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.190195 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.190406 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.190407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.190408 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.190488 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.190795 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.190796 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.190854 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.190951 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.191061 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 14:35:13.191639 18374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:13.191998 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.191889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47646)
0811 14:35:13.191942 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.191975 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 14:35:13.192355 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.192295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47652)
0811 14:35:13.192322 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.192342 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260811 14:35:13.192466 18374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:46639: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
I20260811 14:35:13.194741 18459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } attempt: 1
I20260811 14:35:13.195261 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.195111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47660)
0811 14:35:13.195177 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.195221 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 14:35:13.195659 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.195538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47664)
0811 14:35:13.195593 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.195629 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 14:35:13.195801 18459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:46639: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
I20260811 14:35:13.223619 18374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } attempt: 1
I20260811 14:35:13.224270 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.224073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47678)
0811 14:35:13.224189 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.224237 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 14:35:13.224735 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.224611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47686)
0811 14:35:13.224656 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.224683 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 14:35:13.224872 18374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:46639: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
W20260811 14:35:13.242610 18467 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 14:35:13.242823 18467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 14:35:13.242842 18467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 14:35:13.242862 18467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 14:35:13.244105 18467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 14:35:13.244145 18467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 14:35:13.244159 18467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 14:35:13.244172 18467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 14:35:13.244220 18467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 14:35:13.245870 18467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.23.186.212:35459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.23.186.254:44751,127.23.186.253:36381,127.23.186.252:46639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/krb5kdc/kudu_127.23.186.252.keytab
--principal=kudu/127.23.186.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.23.186.252:46639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.23.186.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 11 Aug 2026 13:41:36 UTC on ccycloud.quasar-ebruqk.root.comops.site
build id 4580675
I20260811 14:35:13.246135 18467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 14:35:13.246369 18467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 14:35:13.251430 18476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:13.251502 18474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 14:35:13.251415 18478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 14:35:13.251775 18467 server_base.cc:1034] running on GCE node
I20260811 14:35:13.252038 18467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 14:35:13.252339 18467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 14:35:13.253527 18467 hybrid_clock.cc:648] HybridClock initialized: now 1786458913253501 us; error 36 us; skew 500 ppm
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 14:35:13.256248 18467 init.cc:377] Logged in from keytab as kudu/127.23.186.252@KRBTEST.COM (short username kudu)
I20260811 14:35:13.256652 18467 webserver.cc:460] Webserver started at http://127.23.186.252:35441/ using document root <none> and password file <none>
I20260811 14:35:13.256924 18467 fs_manager.cc:359] Metadata directory not provided
I20260811 14:35:13.256973 18467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 14:35:13.257063 18467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 14:35:13.258108 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data/instance:
uuid: "cf5007fff2b944beaa5ac4abd54680fc"
format_stamp: "Formatted at 2026-08-11 14:35:13 on dist-test-slave-cbsf"
server_key: "d10dd08eff95a8e9983d33e670629ffb"
server_key_iv: "953cd497fb44bd96c3a0238f128016cf"
server_key_version: "encryptionkey@0"
I20260811 14:35:13.258430 18467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal/instance:
uuid: "cf5007fff2b944beaa5ac4abd54680fc"
format_stamp: "Formatted at 2026-08-11 14:35:13 on dist-test-slave-cbsf"
server_key: "d10dd08eff95a8e9983d33e670629ffb"
server_key_iv: "953cd497fb44bd96c3a0238f128016cf"
server_key_version: "encryptionkey@0"
I20260811 14:35:13.259675 18467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260811 14:35:13.260352 18486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 14:35:13.260541 18467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260811 14:35:13.260637 18467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data,/tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal
uuid: "cf5007fff2b944beaa5ac4abd54680fc"
format_stamp: "Formatted at 2026-08-11 14:35:13 on dist-test-slave-cbsf"
server_key: "d10dd08eff95a8e9983d33e670629ffb"
server_key_iv: "953cd497fb44bd96c3a0238f128016cf"
server_key_version: "encryptionkey@0"
I20260811 14:35:13.260739 18467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 14:35:13.263970 18459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } attempt: 2
I20260811 14:35:13.264394 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.264261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47696)
0811 14:35:13.264325 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.264363 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 14:35:13.264765 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.264665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47712)
0811 14:35:13.264686 (+ 21us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.264746 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260811 14:35:13.264875 18459 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.23.186.252:46639: Network error: Client connection negotiation failed: client connection to 127.23.186.252:46639: connect: Connection refused (error 111)
I20260811 14:35:13.278852 18467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 14:35:13.281248 18467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 14:35:13.281435 18467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 14:35:13.285574 18467 rpc_server.cc:307] RPC server started. Bound to: 127.23.186.252:46639
I20260811 14:35:13.285652 18542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.23.186.252:46639 every 8 connection(s)
I20260811 14:35:13.286001 18467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/data/info.pb
I20260811 14:35:13.286984 18543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 14:35:13.288489 18543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:13.292264 24299 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLI13IK/build/release/bin/kudu as pid 18467
I20260811 14:35:13.292400 24299 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLI13IK/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786458616937656-24299-0/minicluster-data/master-2/wal/instance
I20260811 14:35:13.292613 24299 external_mini_cluster.cc:1428] Setting key fb27faa4d5bf82c3b21719cc5a48b5d1
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:13.297720 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.289258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33618 (local address 127.23.186.254:44751)
0811 14:35:13.289328 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.289330 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.289521 (+ 191us) server_negotiation.cc:407] Connection header received
0811 14:35:13.290018 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.290019 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.290029 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.290086 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.290652 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.291117 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.292065 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.292246 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.292299 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.294653 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.294655 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.294656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.294663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.296158 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.296710 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.296712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.296712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.296777 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.297054 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.297055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.297055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.297189 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.297262 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.297577 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.297656 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 14:35:13.297960 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.289214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.254:44751 (local address 127.0.0.1:33618)
0811 14:35:13.289391 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.289433 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.289927 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.290136 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.290139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.290381 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.290581 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.290585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.291199 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.291200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.291987 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.291989 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.292346 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.292348 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.292520 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.293178 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.293188 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.294552 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.296264 (+ 1712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.296271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.296273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.296586 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.296848 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.296849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.296849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.296975 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.297265 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.297273 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.297414 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.297545 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.297703 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:13.298635 18543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } has no permanent_uuid. Determining permanent_uuid...
I20260811 14:35:13.300045 18374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } attempt: 2
I20260811 14:35:13.301813 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.293066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47728 (local address 127.23.186.252:46639)
0811 14:35:13.293246 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.293251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.293277 (+ 26us) server_negotiation.cc:407] Connection header received
0811 14:35:13.293303 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.293305 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.293326 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.293420 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.294051 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.294950 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.295716 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.296029 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.296118 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.298633 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.298641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.298642 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.298648 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.300519 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.300964 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.300967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.300968 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.301029 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.301219 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.301219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.301220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.301389 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.301558 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.301633 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.301712 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.254@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:13.306679 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.298977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51128 (local address 127.23.186.253:36381)
0811 14:35:13.299020 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.299022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.299030 (+ 8us) server_negotiation.cc:407] Connection header received
0811 14:35:13.299055 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.299056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.299068 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.299126 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.299439 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.300177 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.301030 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.301354 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.301443 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.303794 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.303795 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.303796 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.303803 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.305307 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.305751 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.305753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.305753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.305817 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.306045 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.306046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.306047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.306202 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.306247 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.306467 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.306559 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260811 14:35:13.306696 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.298890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.253:36381 (local address 127.0.0.1:51128)
0811 14:35:13.298951 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.298960 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.298994 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.299143 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.299144 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.299307 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.299352 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.299353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.300254 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.300255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.300904 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.300907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.301463 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.301464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.301536 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.302219 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.302225 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.303695 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.305427 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.305428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.305429 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.305658 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.305890 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.305892 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.305892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.305952 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.306300 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.306301 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.306346 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.306427 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.306578 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 14:35:13.307152 18543 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.254@KRBTEST.COM
I20260811 14:35:13.309487 18377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.300291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47732)
0811 14:35:13.300343 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.300355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.300393 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.300774 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.300775 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.300931 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.300976 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.300977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.302244 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.302246 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.303084 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.303087 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.303615 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.303616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.303668 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.304068 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.304073 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.305653 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.308363 (+ 2710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.308364 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.308364 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.308590 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.308849 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.308850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.308851 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.308920 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.309204 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.309205 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.309249 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.309296 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.309389 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 14:35:13.310119 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.300476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.23.186.252:46639)
0811 14:35:13.300628 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.300630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.300636 (+ 6us) server_negotiation.cc:407] Connection header received
0811 14:35:13.300655 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.300655 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.300665 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.300755 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.301501 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.302129 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.303171 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.303515 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.303613 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.305747 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.305747 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.305748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.305755 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.308242 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.308671 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.308672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.308672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.308770 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.308992 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.308993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.308993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.309123 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.309859 (+ 736us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.309913 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.309990 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.252@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:13.312201 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.302848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33632 (local address 127.23.186.254:44751)
0811 14:35:13.302883 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.302885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.302892 (+ 7us) server_negotiation.cc:407] Connection header received
0811 14:35:13.302912 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.302913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.302921 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.302978 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.303548 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.304891 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.305672 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.305954 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.306451 (+ 497us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.309041 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.309042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.309042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.309048 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.310568 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.310996 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.310997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.310998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.311063 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.311255 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.311256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.311257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.311384 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.311985 (+ 601us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.312028 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.312121 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260811 14:35:13.312358 18374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3: Bootstrap starting.
I20260811 14:35:13.313151 18374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:13.313476 18374 log.cc:824] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:13.314335 18374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3: No bootstrap required, opened a new log
I20260811 14:35:13.315116 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.307727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47746 (local address 127.23.186.252:46639)
0811 14:35:13.307765 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.307767 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.307806 (+ 39us) server_negotiation.cc:407] Connection header received
0811 14:35:13.307867 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.307868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.307879 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.307941 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.308296 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.309004 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.309893 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.310180 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.310261 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.312117 (+ 1856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.312118 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.312118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.312125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.313971 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.314288 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.314289 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.314289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.314349 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.314556 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.314557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.314557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.314679 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.314730 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.314917 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.315025 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":17}
I20260811 14:35:13.315121 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.307694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47746)
0811 14:35:13.307764 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.307774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.307818 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.307947 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.307948 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.308177 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.308212 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.308214 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.309079 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.309079 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.309816 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.309820 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.310272 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.310273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.310340 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.310709 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.310714 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.312035 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.314057 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.314058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.314059 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.314211 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.314431 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.314432 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.314432 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.314493 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.314750 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.314751 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.314797 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.314881 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.314982 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 14:35:13.316064 18374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.316181 18374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d7a2ed068f642aa91e1be8a8b55b0d3, State: Initialized, Role: FOLLOWER
I20260811 14:35:13.316263 18374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.316318 18543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: Bootstrap starting.
I20260811 14:35:13.316524 18552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.316576 18374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:13.317101 18543 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:13.317448 18543 log.cc:824] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:13.318168 18552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:35:13.318764 18566 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:13.318817 18566 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:13.319752 18543 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: No bootstrap required, opened a new log
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.23.186.253@KRBTEST.COM
I20260811 14:35:13.322155 18543 raft_consensus.cc:348] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.322304 18543 raft_consensus.cc:729] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf5007fff2b944beaa5ac4abd54680fc, State: Initialized, Role: FOLLOWER
I20260811 14:35:13.322381 18543 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.322697 18543 sys_catalog.cc:564] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:13.319877 18299 master_runner.cc:186] Error getting master registration for 127.23.186.253:36381: OK, instance_id { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" instance_seqno: 1786458913120205 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:35:13.323376 18299 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:35:13.323520 18568 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.323616 18568 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:35:13.324975 18580 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:13.325032 18580 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:13.325297 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.313804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.23.186.253:36381)
0811 14:35:13.313846 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.313848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.313858 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:13.313880 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.313881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.313891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.313950 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.317188 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.317843 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.318773 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.319053 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.319118 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.321813 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.321814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.321815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.321823 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.323638 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.324090 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.324091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.324091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.324154 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.324346 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.324347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.324347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.324478 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.324666 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.325119 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.325197 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:13.326892 24299 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 14:35:13.327798 18345 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33632
I20260811 14:35:13.327900 18345 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 14:35:13.327942 18345 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:13.326535 18467 master_runner.cc:186] Error getting master registration for 127.23.186.253:36381: OK, instance_id { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" instance_seqno: 1786458913120205 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 14:35:13.329022 18467 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 14:35:13.329166 18345 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.329447 18345 leader_election.cc:288] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [CANDIDATE]: Term 1 election: Requested vote from peers 8a15426942c74ff1ae1ecaeba53ee87e (127.23.186.253:36381), cf5007fff2b944beaa5ac4abd54680fc (127.23.186.252:46639)
I20260811 14:35:13.329665 18514 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "cf5007fff2b944beaa5ac4abd54680fc"
I20260811 14:35:13.329770 18514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:13.330653 18514 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d7a2ed068f642aa91e1be8a8b55b0d3 in term 1.
I20260811 14:35:13.330911 18318 leader_election.cc:302] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d7a2ed068f642aa91e1be8a8b55b0d3, cf5007fff2b944beaa5ac4abd54680fc; no voters:
I20260811 14:35:13.331048 18552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 14:35:13.331213 18552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [term 1 LEADER]: Becoming Leader. State: Replica: 3d7a2ed068f642aa91e1be8a8b55b0d3, State: Running, Role: LEADER
I20260811 14:35:13.331308 18552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.331734 18552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7a2ed068f642aa91e1be8a8b55b0d3. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.331830 18552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:13.336822 18595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 14:35:13.337051 18595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 14:35:13.338569 18378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.330526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.23.186.254:44751)
0811 14:35:13.330569 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.330571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.330910 (+ 339us) server_negotiation.cc:407] Connection header received
0811 14:35:13.331067 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.331083 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.331097 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.331130 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.331936 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.332656 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.333712 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.333910 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.333991 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.335504 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.335505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.335505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.335512 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.337260 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.337665 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.337666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.337666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.337739 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.337959 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.337959 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.337960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.338086 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.338201 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.338390 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.338466 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260811 14:35:13.344177 18514 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [term 1 FOLLOWER]: Refusing update from remote peer 3d7a2ed068f642aa91e1be8a8b55b0d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:13.344532 18552 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:13.345881 18568 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7a2ed068f642aa91e1be8a8b55b0d3. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.345988 18568 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:13.346748 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.330755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.23.186.252:46639)
0811 14:35:13.330837 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.330839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.338693 (+ 7854us) server_negotiation.cc:407] Connection header received
0811 14:35:13.338787 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.338789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.338799 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.338828 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.339474 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.339944 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.340950 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.341156 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.341237 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.342573 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.342574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.342575 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.342582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.344890 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.345504 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.345506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.345519 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.345582 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.345926 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.345927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.345927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.346079 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.346155 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.346535 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.346635 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 14:35:13.348476 18568 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.348474 18552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.348558 18568 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:13.348560 18552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:13.349200 18463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.330966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.23.186.253:36381)
0811 14:35:13.331008 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.331010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.340858 (+ 9848us) server_negotiation.cc:407] Connection header received
0811 14:35:13.340955 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.340957 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.340969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.341000 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.341771 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.342541 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.343530 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.343868 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.343939 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.345605 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.345606 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.345607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.345615 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.347446 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.348142 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.348144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.348144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.348209 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.348440 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.348441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.348441 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.348581 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.348658 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.349006 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.349088 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260811 14:35:13.349715 18596 mvcc.cc:205] Tried to move back new op lower bound from 7317335709055168512 to 7317335709005803520. Current Snapshot: MvccSnapshot[applied={T|T < 7317335709055168512}]
I20260811 14:35:13.356006 18459 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } attempt: 3
I20260811 14:35:13.356962 18597 mvcc.cc:205] Tried to move back new op lower bound from 7317335709055168512 to 7317335709005803520. Current Snapshot: MvccSnapshot[applied={T|T < 7317335709055168512}]
I20260811 14:35:13.357566 18595 catalog_manager.cc:1347] Generated new cluster ID: f35d89f05a9247348e1d5d32bbc69e2c
I20260811 14:35:13.357618 18595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 11 14:35:13 dist-test-slave-cbsf krb5kdc[18283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786458913, etypes {rep=17 tkt=17 ses=17}, kudu/127.23.186.253@KRBTEST.COM for kudu/127.23.186.252@KRBTEST.COM
I20260811 14:35:13.365164 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.356288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.23.186.252:46639 (local address 127.0.0.1:47764)
0811 14:35:13.356341 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0811 14:35:13.356352 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 14:35:13.356388 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 14:35:13.356526 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 14:35:13.356528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 14:35:13.356686 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 14:35:13.356790 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.356792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.357692 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.357693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.358492 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 14:35:13.358495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.358970 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.358971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 14:35:13.359025 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.359877 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 14:35:13.359882 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 14:35:13.361454 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 14:35:13.364010 (+ 2556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.364011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.364012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.364244 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.364458 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 14:35:13.364459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 14:35:13.364459 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0811 14:35:13.364517 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 14:35:13.364813 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 14:35:13.364814 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 14:35:13.364862 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 14:35:13.364960 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 14:35:13.365065 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 14:35:13.365164 18548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:13.356345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47764 (local address 127.23.186.252:46639)
0811 14:35:13.356384 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.356386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.356395 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:13.356441 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.356441 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.356453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.356513 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.356885 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.357580 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.358588 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:13.358896 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:13.358949 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:13.361560 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 14:35:13.361561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 14:35:13.361562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 14:35:13.361569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0811 14:35:13.363933 (+ 2364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.364327 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.364328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.364329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.364398 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 14:35:13.364570 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 14:35:13.364571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 14:35:13.364571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 14:35:13.364752 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 14:35:13.364816 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:13.364988 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 14:35:13.365065 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260811 14:35:13.367208 18459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: Bootstrap starting.
I20260811 14:35:13.367995 18459 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: Neither blocks nor log segments found. Creating new log.
I20260811 14:35:13.368319 18459 log.cc:824] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: Log is configured to *not* fsync() on all Append() calls
I20260811 14:35:13.369225 18459 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: No bootstrap required, opened a new log
I20260811 14:35:13.371336 18459 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.371488 18459 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a15426942c74ff1ae1ecaeba53ee87e, State: Initialized, Role: FOLLOWER
I20260811 14:35:13.371578 18459 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } }
I20260811 14:35:13.371961 18459 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 14:35:13.372805 18429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [term 0 FOLLOWER]: Advancing to term 1
I20260811 14:35:13.372804 18430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8a15426942c74ff1ae1ecaeba53ee87e"
I20260811 14:35:13.373724 18429 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [term 1 FOLLOWER]: Refusing update from remote peer 3d7a2ed068f642aa91e1be8a8b55b0d3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 14:35:13.374661 18430 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d7a2ed068f642aa91e1be8a8b55b0d3 in term 1.
I20260811 14:35:13.375105 18601 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 14:35:13.375579 18604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.375692 18604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:13.377200 18622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d7a2ed068f642aa91e1be8a8b55b0d3. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.377269 18622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 14:35:13.377300 18622 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.377349 18622 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e [sys.catalog]: This master's current role is: FOLLOWER
W20260811 14:35:13.378139 18617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 14:35:13.379096 18617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 14:35:13.378607 18619 mvcc.cc:205] Tried to move back new op lower bound from 7317335709055168512 to 7317335709005803520. Current Snapshot: MvccSnapshot[applied={T|T < 7317335709055168512}]
I20260811 14:35:13.389866 18591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d7a2ed068f642aa91e1be8a8b55b0d3" member_type: VOTER last_known_addr { host: "127.23.186.254" port: 44751 } } peers { permanent_uuid: "8a15426942c74ff1ae1ecaeba53ee87e" member_type: VOTER last_known_addr { host: "127.23.186.253" port: 36381 } } peers { permanent_uuid: "cf5007fff2b944beaa5ac4abd54680fc" member_type: VOTER last_known_addr { host: "127.23.186.252" port: 46639 } } }
I20260811 14:35:13.390005 18591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3 [sys.catalog]: This master's current role is: LEADER
I20260811 14:35:13.392975 18595 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 14:35:13.393733 18595 catalog_manager.cc:1504] Loading token signing keys...
I20260811 14:35:13.399381 18595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d7a2ed068f642aa91e1be8a8b55b0d3: Generated new TSK 0
I20260811 14:35:13.399573 18595 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 14:35:13.427976 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.426421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51158 (local address 127.23.186.253:36381)
0811 14:35:13.426472 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.426474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.427220 (+ 746us) server_negotiation.cc:407] Connection header received
0811 14:35:13.427337 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.427339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.427351 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.427383 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.427896 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51158: BlockingRecv error: recv got EOF from 127.0.0.1:51158 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260811 14:35:13.428815 18378 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.426870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33664 (local address 127.23.186.254:44751)
0811 14:35:13.426927 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.426929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.428248 (+ 1319us) server_negotiation.cc:407] Connection header received
0811 14:35:13.428350 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.428352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.428362 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.428392 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.428750 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33664: BlockingRecv error: recv got EOF from 127.0.0.1:33664 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 14:35:13.429898 18548 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.429198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.23.186.252:46639)
0811 14:35:13.429255 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.429257 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.429344 (+ 87us) server_negotiation.cc:407] Connection header received
0811 14:35:13.429446 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.429447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.429459 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.429490 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.429819 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47770: BlockingRecv error: recv got EOF from 127.0.0.1:47770 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 14:35:13.432401 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.431601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51170 (local address 127.23.186.253:36381)
0811 14:35:13.431639 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.431641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.431832 (+ 191us) server_negotiation.cc:407] Connection header received
0811 14:35:13.431965 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.431966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.431988 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.432019 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.432353 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51170: BlockingRecv error: recv got EOF from 127.0.0.1:51170 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260811 14:35:13.435540 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.434714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.23.186.253:36381)
0811 14:35:13.434748 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.434750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.435023 (+ 273us) server_negotiation.cc:407] Connection header received
0811 14:35:13.435120 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.435121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.435132 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.435161 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.435488 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51176: BlockingRecv error: recv got EOF from 127.0.0.1:51176 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260811 14:35:13.439468 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.438622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.23.186.253:36381)
0811 14:35:13.438660 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.438662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.438923 (+ 261us) server_negotiation.cc:407] Connection header received
0811 14:35:13.439038 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.439039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.439051 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.439083 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.439421 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51180: BlockingRecv error: recv got EOF from 127.0.0.1:51180 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260811 14:35:13.445729 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.444753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.23.186.253:36381)
0811 14:35:13.444794 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.444796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.445147 (+ 351us) server_negotiation.cc:407] Connection header received
0811 14:35:13.445254 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.445255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.445267 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.445297 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.445675 (+ 378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51194: BlockingRecv error: recv got EOF from 127.0.0.1:51194 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260811 14:35:13.456061 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.455054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51202 (local address 127.23.186.253:36381)
0811 14:35:13.455104 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.455106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.455416 (+ 310us) server_negotiation.cc:407] Connection header received
0811 14:35:13.455530 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.455531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.455544 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.455576 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.455986 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51202: BlockingRecv error: recv got EOF from 127.0.0.1:51202 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260811 14:35:13.474565 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.473504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51218 (local address 127.23.186.253:36381)
0811 14:35:13.473552 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.473555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.473901 (+ 346us) server_negotiation.cc:407] Connection header received
0811 14:35:13.474028 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.474029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.474044 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.474075 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.474502 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51218: BlockingRecv error: recv got EOF from 127.0.0.1:51218 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260811 14:35:13.508750 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.507941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.23.186.253:36381)
0811 14:35:13.507991 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.507993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.508149 (+ 156us) server_negotiation.cc:407] Connection header received
0811 14:35:13.508221 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.508222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.508234 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.508309 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.508665 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51228: BlockingRecv error: recv got EOF from 127.0.0.1:51228 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260811 14:35:13.574636 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.573863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51234 (local address 127.23.186.253:36381)
0811 14:35:13.573938 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.573940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.574047 (+ 107us) server_negotiation.cc:407] Connection header received
0811 14:35:13.574115 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.574116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.574130 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.574206 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.574566 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51234: BlockingRecv error: recv got EOF from 127.0.0.1:51234 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 14:35:13.704608 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.703897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51250 (local address 127.23.186.253:36381)
0811 14:35:13.703982 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.703984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.704031 (+ 47us) server_negotiation.cc:407] Connection header received
0811 14:35:13.704094 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.704095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.704114 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.704158 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.704524 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51250: BlockingRecv error: recv got EOF from 127.0.0.1:51250 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 14:35:13.962405 18463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 14:35:13.961804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.23.186.253:36381)
0811 14:35:13.961859 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:13.961861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:13.961871 (+ 10us) server_negotiation.cc:407] Connection header received
0811 14:35:13.961914 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:13.961914 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:13.961925 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:13.961974 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 14:35:13.962326 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51260: BlockingRecv error: recv got EOF from 127.0.0.1:51260 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
I20260811 14:35:14.325498 18580 catalog_manager.cc:1259] Loaded cluster ID: f35d89f05a9247348e1d5d32bbc69e2c
I20260811 14:35:14.325580 18580 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: loading cluster ID for follower catalog manager: success
I20260811 14:35:14.326435 18580 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: acquiring CA information for follower catalog manager: success
I20260811 14:35:14.326762 18580 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cf5007fff2b944beaa5ac4abd54680fc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:35:14.380091 18617 catalog_manager.cc:1259] Loaded cluster ID: f35d89f05a9247348e1d5d32bbc69e2c
I20260811 14:35:14.380183 18617 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: loading cluster ID for follower catalog manager: success
I20260811 14:35:14.381042 18617 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: acquiring CA information for follower catalog manager: success
I20260811 14:35:14.381319 18617 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8a15426942c74ff1ae1ecaeba53ee87e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 14:35:14.478329 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:14.475697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51270 (local address 127.23.186.253:36381)
0811 14:35:14.475831 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:14.475833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:14.475874 (+ 41us) server_negotiation.cc:407] Connection header received
0811 14:35:14.475936 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:14.475937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:14.476002 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:14.476091 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:14.476245 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.476815 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.477597 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.477797 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.477854 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:14.478015 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:14.478134 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:14.478195 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:14.478240 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 14:35:14.478259 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260811 14:35:14.482625 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:14.480169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47772 (local address 127.23.186.252:46639)
0811 14:35:14.480301 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:14.480304 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:14.480313 (+ 9us) server_negotiation.cc:407] Connection header received
0811 14:35:14.480367 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:14.480368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:14.480407 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:14.480495 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:14.480586 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.481133 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.481868 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.482062 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.482113 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:14.482276 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:14.482382 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:14.482438 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:14.482540 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 14:35:14.482559 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260811 14:35:14.487964 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:14.484714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47786 (local address 127.23.186.252:46639)
0811 14:35:14.484818 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:14.484819 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:14.484978 (+ 159us) server_negotiation.cc:407] Connection header received
0811 14:35:14.485041 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:14.485042 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:14.485054 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:14.485084 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:14.485273 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.486058 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.487187 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.487466 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.487682 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:14.487706 (+ 24us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:14.487741 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:14.487790 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:14.487839 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 14:35:14.487864 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 14:35:14.488086 18695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:14.484636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51286 (local address 127.23.186.253:36381)
0811 14:35:14.484787 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:14.484790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:14.485468 (+ 678us) server_negotiation.cc:407] Connection header received
0811 14:35:14.485576 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:14.485577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:14.485590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:14.485622 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:14.485827 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.486461 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.487247 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.487563 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.487624 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:14.487775 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:14.487810 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:14.487858 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:14.487966 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 14:35:14.487999 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 14:35:14.488325 18708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 14:35:14.484531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33668 (local address 127.23.186.254:44751)
0811 14:35:14.484872 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 14:35:14.484874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0811 14:35:14.484885 (+ 11us) server_negotiation.cc:407] Connection header received
0811 14:35:14.484936 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 14:35:14.484937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 14:35:14.484993 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 14:35:14.485261 (+ 268us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 14:35:14.485311 (+ 50us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.486103 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.487183 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 14:35:14.487436 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 14:35:14.487520 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 14:35:14.487729 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 14:35:14.487861 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 14:35:14.488181 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 14:35:14.488204 (+ 23us) server_negotiation.cc:299] Negotiation successful
0811 14:35:14.488233 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":74,"threads_started":1}
I20260811 14:35:14.490144 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 18299
I20260811 14:35:14.499329 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 18382
I20260811 14:35:14.506996 24299 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLI13IK/build/release/bin/kudu with pid 18467
2026-08-11T14:35:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2975 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5995 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (297580 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST